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This tutorial assumes no previous knowledge of scripting or programming, but progresses rapidly toward an 
intermediate/advanced level of instruction . . . all the while sneaking in little nuggets of UNIX® wisdom and 
lore. It serves as a textbook, a manual for self-study, and a reference and source of knowledge on shell 
scripting techniques. The exercises and heavily-commented examples invite active reader participation, under 
the premise thatthe only way to really learn scripting is to write scripts. 


This book is suitable for classroom use as a general introduction to programming concepts. 
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Part 1. Introduction 


Script: A writing; a written document. [Obs.] 
--Webster's Dictionary, 1913 ed. 


The shell is a command interpreter. More than just the insulating layer between the operating system kernel 
and the user, it's also a fairly powerful programming language. A shell program, called a script, is an 
easy-to-use tool for building applications by "gluing together" system calls, tools, utilities, and compiled 
binaries. Virtually the entire repertoire of UNIX commands, utilities, and tools is available for invocation by a 
shell script. If that were not enough, internal shell commands, such as testing and loop constructs, lend 
additional power and flexibility to scripts. Shell scripts are especially well suited for administrative system 
tasks and other routine repetitive tasks not requiring the bells and whistles of a full-blown tightly structured 
programming language. 


Table of Contents 


1. Shell Programming! 
2. Starting Off With a Sha-Bang 


Prev Home Next 
Advanced Bash-Scripting Guide Shell Programming! 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Next 


Chapter 1. Shell Programming! 


No programming language is perfect. There is 
not even a single best language; there are only 
languages well suited or perhaps poorly suited 
for particular purposes. 


--Herbert Mayer 
A working knowledge of shell scripting is essential to anyone wishing to become reasonably proficient at 
system administration, even if they do not anticipate ever having to actually write a script. Consider that as a 
Linux machine boots up, it executes the shell scripts in /etc/rc.d to restore the system configuration and 
set up services. A detailed understanding of these startup scripts is important for analyzing the behavior of a 
system, and possibly modifying it. 


The craft of scripting is not hard to master, since the scripts can be built in bite-sized sections and there is only 
a fairly small set of shell-specific operators and options [1] to learn. The syntax is simple and straightforward, 
similar to that of invoking and chaining together utilities at the command line, and there are only a few "rules" 
governing their use. Most short scripts work right the first time, and debugging even the longer ones is 
straightforward. 


In the 1970s, the BASIC language enabled anyone reasonably computer proficient 

to write programs on an early generation of microcomputers. Decades later, the Bash 

scripting language enables anyone with a rudimentary knowledge of Linux or UNIX to do the same 
on much more powerful machines. 


A shell script is a quick-and-dirty method of prototyping a complex application. Getting even a limited subset 
of the functionality to work in a script is often a useful first stage in project development. This way, the 
structure of the application can be tested and played with, and the major pitfalls found before proceeding to 
the final coding in C, C++, Java, Perl, or Python. 


Shell scripting hearkens back to the classic UNIX philosophy of breaking complex projects into simpler 
subtasks, of chaining together components and utilities. Many consider this a better, or at least more 
esthetically pleasing approach to problem solving than using one of the new generation of high powered 
all-in-one languages, such as Perl, which attempt to be all things to all people, but at the cost of forcing you to 
alter your thinking processes to fit the tool. 


According to Herbert Mayer, "a useful language needs arrays, pointers, and a generic mechanism for building 
data structures." By these criteria, shell scripting falls somewhat short of being "useful." Or, perhaps not. . . . 


When not to use shell scripts 


* Resource-intensive tasks, especially where speed is a factor (sorting, hashing, recursion [2] ...) 

* Procedures involving heavy-duty math operations, especially floating point arithmetic, arbitrary 
precision calculations, or complex numbers (use C++ or FORTRAN instead) 

* Cross-platform portability required (use C or Java instead) 

* Complex applications, where structured programming is a necessity (type-checking of variables, 
function prototypes, etc.) 

* Mission-critical applications upon which you are betting the future of the company 


e Situations where security is important, where you need to guarantee the integrity of your system and 
protect against intrusion, cracking, and vandalism 

* Project consists of subcomponents with interlocking dependencies 

* Extensive file operations required (Bash is limited to serial file access, and that only in a 
particularly clumsy and inefficient line-by-line fashion.) 

* Need native support for multi-dimensional arrays 

* Need data structures, such as linked lists or trees 

* Need to generate / manipulate graphics or GUIs 

* Need direct access to system hardware 

* Need port or socket I/O 

* Need to use libraries or interface with legacy code 

* Proprietary, closed-source applications (Shell scripts put the source code right out in the open for all 
the world to see.) 


If any of the above applies, consider a more powerful scripting language -- perhaps Perl, Tcl, Python, Ruby 
-- or possibly a compiled language such as C, C++, or Java. Even then, prototyping the application as a 
shell script might still be a useful development step. 


We will be using Bash, an acronym for "Bourne-Again shell" and a pun on Stephen Bourne's now classic 
Bourne shell. Bash has become a de facto standard for shell scripting on most flavors of UNIX. Most of the 
principles this book covers apply equally well to scripting with other shells, such as the Korn Shell, from 
which Bash derives some of its features, [3] and the C Shell and its variants. (Note that C Shell programming 
is not recommended due to certain inherent problems, as pointed out in an October, 1993 Usenet post by Tom 
Christiansen.) 


What follows is a tutorial on shell scripting. It relies heavily on examples to illustrate various features of the 
shell. The example scripts work -- they've been tested, insofar as was possible -- and some of them are even 
useful in real life. The reader can play with the actual working code of the examples in the source archive 
(scriptname.shorscriptname.bashb),[4]give them execute permission (chmod u+rx 
scriptname), then run them to see what happens. Should the source archive not be available, then 
cut-and-paste from the HTML or pdf rendered versions. Be aware that some of the scripts presented here 
introduce features before they are explained, and this may require the reader to temporarily skip ahead for 
enlightenment. 


Unless otherwise noted, the author of this book wrote the example scripts that follow. 
His countenance was bold and bashed not. 


--Edmund Spenser 
Notes 


[1] These are referred to as builtins, features internal to the shell. 


[2] Although recursion is possible in a shell script, it tends to be slow and its implementation is often an 
ugly kludge. 

[3] Many of the features of ksh68, and even a few from the updated ksh93 have been merged into Bash. 

[4] By convention, user-written shell scripts that are Bourne shell compliant generally take a name with a 


. sh extension. System scripts, such as those found in /et c/rc.d, do not conform to this 
nomenclature. 
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Chapter 2. Starting Off With a Sha-Bang 


Shell programming is a 1950s juke box... 


--Larry Wall 
In the simplest case, a script is nothing more than a list of system commands stored in a file. At the very least, 
this saves the effort of retyping that particular sequence of commands each time it is invoked. 


Example 2-1. cleanup: A script to clean up the log files in /var/log 


# Cleanup 
# Run as root, of course. 


cat /dev/null » messages 
cat /dev/null » wtmp 


il 
2 
E 
4 cd /var/log 
5 
6 
7 echo "Logs cleaned up." 


There is nothing unusual here, only a set of commands that could just as easily have been invoked one by one 
from the command-line on the console or in a terminal window. The advantages of placing the commands in a 
script go far beyond not having to retype them time and again. The script becomes a program -- a tool -- and it 
can easily be modified or customized for a particular application. 


Example 2-2. cleanup: An improved clean-up script 


!/bin/bash 
Proper header for a Bash script. 


Cleanup, version 2 


Run as root, of course. 
Insert code here to print error message and exit if not root. 


LOG DIR-/var/log 
Variables are better than hard-coded values. 
(exl Sie DIE 


cat /dev/null » messages 
cat /dev/null » wtmp 


echo "Logs cleaned up." 


Wey tee] (ex, (On dex. (GS) (SS) [= «c» Wer (o9) SS) py (Oa des (5 de». [23 


exit 4 The right and proper method of "exiting" from a script. 


Now that's beginning to look like a real script. But we can go even farther . . . 


Example 2-3. cleanup: An enhanced and generalized version of above scripts. 


#!/bin/bash 
# Cleanup, version 3 


# n T 


1 
2 
3 
4 # Warning: 
5 
6 # This script uses quite a number of features that will be explained 


NO +t H H H H H H H = f 
@) Se (oer c] y Gal PSs CO (IS) [| Sp Ke) Co) cu 


PAES ISS) [Bor RS) Aber SY TSS) ISS) ner des) 
|. «e» We) (Ger 3) ony (Gal des We) [Sy dS 


WWWWW CO CO CO 
Wey (oer d] (ox Gal ges (63 IS) 


op 
= €» 


CO) Pf bb SP Am SS 
(Gy Wer sr p von) Gal HSS CS) fe») 


or En 
NO ES 


(Oni Gal ton) Xn] Gah (Gui: (nl 
«oO Kes) ss] O) O1 iS CO 


py tex TO) Xen Ly OE Top) 
oy Gal de fs) [Sy [3M 6» 


-1O0o0 0 
(ey (ex (eor | 


- - 
ISS) f= 


#+ later on. 


#+ there should be nothing mysterious about it. 


LOG DIR-/var/log 
ROOT UID-0 
,INES=50 
E_XCD=86 

E NOTROOT-87 


4 Default number of lines saved. 
# Can't change directory? 
# Non-root exit error. 


of course. 
USROOMaUMD Uae 


Born ES COGE; 
aie [p VSUDE 
then 
Seine "E loe TAOOIE 19 wu elms Serene.” 
exit SE_NOTROOT 
EL 


IOS 


aa || ew 
# Test whether command-line argument is present 
then 

lines-$1 
else 

lines-S$LINES # Default, 
fti 


Sy 


if not specified onc 


Stephane Chazelas suggests the following, 


awe elias is SEIL @ lone axchvaiacecl itor clans Sic 


E WRONGARGS-85 # Non-numerical argument ( 


Gage WSal aun 

aie ) lines-50;; 

"39-917 echo Visage: basanama SO” File- 
js ) lines=$1;; 

esac 


* Skip ahead to "Loops" chapter to decipher al 


cd $LOG DIR 


aie [D euel Ve VSLOG JUIN J Gp oue ai [p SPD 
# Not in /var/log? 

then 

ceho "Gus iE. change co SLOG DIR, VU 

exit SE XCD 
fi # Doublecheck if in right directory before 
# Far more efficient is: 
# 
u- eol fryarc/log 4 
# echo "Cannot change to necessary directory. 
d e»x3i UE MCs 
ie o» 
tail -n $lines messages > mesg.temp # Save last 
mv mesg.temp messages # Becomes n 


By the time you've finished the first half of the book, 


# Only users with SUID 0 have root privileges. 


(non-empty). 


ommand-line. 


+ as a better way of checking command-line arguments, 


age of the tutorial. 


bad argument format). 


exit $E WRONGARGS;; 


to-cleanup"; 


JL ie loalss 


l= 


WO DIR J 


messing with log file. 


WD 


section of message log file. 
ew log directory. 


73 

74 # cat /dev/null > messages 

75 #* No longer needed, as the above method is safer. 

76 

77 cat /dev/null > wtmp 4$: ': > wtmp' and '» wtmp' have the sam ffect. 
78 echo "Logs cleaned up." 

TS 

80 exit 0 

81 # A zero return value from the script upon exit indicates success 

82 #+ to the shell. 


Since you may not wish to wipe out the entire system log, this version of the script keeps the last section of 
the message log intact. You will constantly discover ways of fine-tuning previously written scripts for 
increased effectiveness. 


x k ok 


The sha-bang ( #!) [1] at the head of a script tells your system that this file is a set of commands to be fed to 
the command interpreter indicated. The #! is actually a two-byte [2] magic number, a special marker that 
designates a file type, or in this case an executable shell script (type man magic for more details on this 
fascinating topic). Immediately following the sha-bang is a path name. This is the path to the program that 
interprets the commands in the script, whether it be a shell, a programming language, or a utility. This 
command interpreter then executes the commands in the script, starting at the top (the line following the 
sha-bang line), and ignoring comments. [3] 


#!/bin/sh 
#!/bin/bash 
#!/usr/bin/perl 
gl /usr/oin/tel 
#!/bin/sed -f 

6 #!/usr/awk -f 
Each of the above script header lines calls a different command interpreter, be it /bin/ sh, the default shell 
(bash in a Linux system) or otherwise. [4] Using #! /bin/sh, the default Bourne shell in most commercial 
variants of UNIX, makes the script portable to non-Linux machines, though you sacrifice Bash-specific 
features. The script will, however, conform to the POSIX [5] sh standard. 


GS Go SS 


Note that the path given at the "sha-bang" must be correct, otherwise an error message -- usually "Command 
not found." -- will be the only result of running the script. [6] 


#! can be omitted if the script consists only of a set of generic system commands, using no internal shell 
directives. The second example, above, requires the initial #!, since the variable assignment line, Lines=50, 
uses a shell-specific construct. [7] Note again that #! /bin/sh invokes the default shell interpreter, which 
defaults to /bin/bash on a Linux machine. 


This tutorial encourages a modular approach to constructing a script. Make note of and collect 
"boilerplate" code snippets that might be useful in future scripts. Eventually you will build quite an 
extensive library of nifty routines. As an example, the following script prolog tests whether the script has 
been invoked with the correct number of parameters. 


E WRONG ARGS-85 
Script parameters-"-a -h -m -z" 
# a all, -h help, etc. 


if [ $4 -ne SNumber_of_expected_args ] 

then 
echo "Usage: 'basename $0' $script parameters" 
# ` basename $0° is the script's filename. 
exit $E WRONG ARGS 


So Cc I Oy Cl m CO INS ES 


dg fx 
Many times, you will write a script that carries out one particular task. The first script in this chapter is 
an example. Later, it might occur to you to generalize the script to do other, similar tasks. Replacing the 
literal ("hard-wired") constants by variables is a step in that direction, as is replacing repetitive code 
blocks by functions. 


2.1. Invoking the script 


Having written the script, you can invoke it by sh scriptname, [8] or alternatively bash scriptname. 
(Not recommended is using sh «scriptname, since this effectively disables reading from st din within 
the script.) Much more convenient is to make the script itself directly executable with a chmod. 


Either: 

chmod 555 scriptname (gives everyone read/execute permission) [9] 
Or 

chmod +rx scriptname (gives everyone read/execute permission) 


chmod u+rx scriptname (gives only the script owner read/execute permission) 


Having made the script executable, you may now test it by ./scriptname. [10] If it begins with a 
"sha-bang" line, invoking the script calls the correct command interpreter to run it. 


As a final step, after testing and debugging, you would likely want to move it to /usr/local/bin (as root, 
of course), to make the script available to yourself and all other users as a systemwide executable. The script 
could then be invoked by simply typing scriptname [ENTER] from the command-line. 


Notes 


[1] Also seen in the literature as she-bang or sh-bang. This derives from the concatenation of the tokens sharp 
(#) and bang (!). 

[2] Some flavors of UNIX (those based on 4.2 BSD) allegedly take a four-byte magic number, requiring a blank 
after the ! -- #! /bin/sh. According to Sven Mascheck this is probably a myth. 


[3] The #! line in a shell script will be the first thing the command interpreter (sh or bash) sees. Since this line 
begins with a #, it will be correctly interpreted as a comment when the command interpreter finally executes 
the script. The line has already served its purpose - calling the command interpreter. 


If, in fact, the script includes an extra #! line, then bash will interpret it as a comment. 


1 #!/bin/bash 
2 
3 Gino Heere dl (xe meras." 
4 a-1 
5 
6 #!/bin/bash 
7 # This does *not* launch a new script. 
8 
9) echo “Waric 2 gut Ies." 
10 echo $a # Value of $a stays at 1. 


[4] This allows some cute tricks. 


1 #!/bin/rm 

2 4 Self-deleting script. 

3 

4 # Nothing much seems to happen when you run this... except that the file disappears. 
E 

6 WHATEVER-85 

7 

8 echo "This line will never print (betcha!) ." 

9 


10 exit SWHATEVER # Doesn't matter. The script will not exit here. 
iil # Try an echo $? after script termination. 
2 # You'll get a 0, not a 85. 


Also, try starting a README file with a #! /bin/more, and making it executable. The result is a self-listing 
documentation file. (A here document using cat is possibly a better alternative -- see Example 19-3). 


Portable Operating System Interface, an attempt to standardize UNIX-like OSes. The POSIX specifications 
are listed on the Open Group site. 

To avoid this possibility, a script may begin with a #!/bin/env bash sha-bang line. This may be useful on 
UNIX machines where bash is not located in /bin 


If Bash is your default shell, then the #! isn't necessary at the beginning of a script. However, if launching a 
script from a different shell, such as tcsh, then you will need the #!. 


therefore fail to execute. 


D] 

[6] 

Ul 

[8] Caution: invoking a Bash script by sh scriptname turns off Bash-specific extensions, and the script may 
[9] A script needs read, as well as execute permission for it to run, since the shell needs to be able to read it. 
uo] 


Why not simply invoke the script with scriptname? If the directory you are in (SPWD) is where 
scriptname is located, why doesn't this work? This fails because, for security reasons, the current 
directory (. /) is not by default included in a user's $PATH. It is therefore necessary to explicitly invoke the 
script in the current directory with a ./scriptname. 
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2.2. Preliminary Exercises 


1. System administrators often write scripts to automate common tasks. Give several instances where 
such scripts would be useful. 

2. Write a script that upon invocation shows the time and date, lists all logged-in users, and gives the 
system uptime. The script then saves this information to a logfile. 


Prev Home Next 
Starting Off With a Sha-Bang Up Basics 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Next 


Part 2. Basics 


Table of Contents 


3. Special Characters 
4. [ntroduction to Variables and Parameters 


5. Quoting 
6. Exit and Exit Status 
7. Tests 


8. Operations and Related Topics 


Prev Home 
Preliminary Exercises 


Z 


ex 
Special Characters 


Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 


Prev 


Z 


ex 


Chapter 3. Special Characters 


What makes a character special? If it has a meaning beyond its literal meaning, a meta-meaning, then we refer 
to it as a special character. 


Special Characters Found In Scripts and Elsewhere 


# 
Comments. Lines beginning with a # (with the exception of #!) are comments and will not be 
executed. 


1 # This line is a comment. 


Comments may also occur following the end of a command. 


1 echo "A comment will follow." # Comment here. 
2 t ^ Note whitespace before # 


Comments may also follow whitespace at the beginning of a line. 


i # A tab precedes this comment. 


Comments may even be embedded within a pipe. 


i aoulivagile( eed UecEeusEisLie | sed e "we || tie sel UN DN 
2 # Delete lines containing '4' comment character. 

3 Geo =e Ys No Xe fg" =e "s/_/ fg ) 

4 4 Excerpted from life.sh script 


A command may not follow a comment on the same line. There is no method of 
terminating the comment, in order for "live code" to begin on the same line. Use a new 
line for the next command. 


cg) Of course, a quoted or an escaped # in an echo statement does not begin a comment. 
Likewise, a # appears in certain parameter-substitution constructs and in numerical 


constant expressions. 


1 echo "The # here does not begin a comment." 

2 echo 'The # here does not begin a comment.' 

3 echo The \# here does not begin a comment. 

4 echo The # here begins a comment. 

5 

6 echo S(PATH£*:) # Parameter substitution, not a comment. 
7 echo $(( 24101011 )) # Base conversion, not a comment. 

8 

9) s wnguolke, SoC. 


The standard quoting and escape characters (" ' V escape the #. 
Certain pattern matching operations also use the #. 


Command separator [semicolon]. Permits putting two or more commands on the same line. 


1 echo hello; cho ther 

2 

3 

4k sb || o VSitnilemene” ja rren # Note the space after the semicolon. 
5 d qos 

6 echo "File $filename exists."; cp $filename $filename.bak 

7 else # eie 

8 echo "File $filename not found."; touch $filename 


9 fi; echo "File test complete." 


Note that the ";" sometimes needs to be escaped. 


33&, 5& 


Terminator in a case option [double semicolon]. 


il ease "SweucseloleV im 

2 abe) eco “\Swaicialole = aloe” pp 
3 Say) acho YA\svariable = y2" p? 
4 esac 


Terminators in a case option (version 4+ of Bash). 


"dot" command [period]. Equivalent to source (see Example 15-22). This is a bash builtin. 


"dot", as a component of a filename. When working with filenames, a leading dot is the prefix of a 
"hidden" file, a file that an Is will not normally show. 


bash$ touch .hidden-file 
bash$ 1s -1 


Togail IL) 

SEWA i 1 bozo 4034 Jul 18 22:04 datal.addressbook 

EW === bozo 4602 May 25 13:58 datal.addressbook.bak 

EW Id l bozo 877 Dec 17 2000 employment.addressbook 
bash$ 1s -al 

total 14 

drwxrwxr-x Zi bozo. bozo 1024 Awe 29 20354 ./ 

CE pi o 5A oOo DOZO 3072 Mug 29 209051 ../ 

—UBW Tdi iL loze bozo 4034 Jul 18 22:04 datal.addressbook 
UBI deo 1 bozo bozo 4602 May 25 13:58 datal.addressbook.bak 
CWE G L loze boza 877 Dec 17 2000 employment.addressbook 
=EN ES L bozo looo 0 Aug 29 20:54 .hidden-file 


When considering directory names, a single dot represents the current working directory, and two dots 
denote the parent directory. 


bash$ pwd 
/home/bozo/projects 


bash$ ed . 

bash$ pwd 
/home/bozo/projects 
bash$ ed .. 


bash$ pwd 
/home/bozo/ 


The dot often appears as the destination (directory) of a file movement command, in this context 
meaning current directory. 


bash$ cp /home/bozo/current work/junk/* 


Copy all the "junk" files to $SPWD. 


"dot" character match. When matching characters, as part of a regular expression, a "dot" matches a 
single character. 


partial quoting [double quote]. "STRING" preserves (from interpretation) most of the special 


characters within STRING. See Chapter 5. 


full quoting [single quote]. ‘STRING’ preserves all special characters within STRING. This is a 
stronger form of quoting than "STRING". See Chapter 5. 


comma operator. The comma operator [1] links together a series of arithmetic operations. All are 
evaluated, but only the last one is returned. 


1 let "t2 = ((a = 9, 15 / 9))" 
2% Sec Va = OY emo Ver = 15 y sw 


The comma operator can also concatenate strings. 


HOw ile dim / 1 ,wsie/ ous “calle 


1 
2 t $i Find all executable files ending in "calc" 
3 #+ in /bin and /usr/bin directories. 
4 do 
Ej aie [| =x VSraile" | 
6 then 
7 echo $file 
8 ial 
9 done 
10 
11 # /bin/ipcalc 
12; s» iens stay keale 
13 # /usr/bin/oidcalc 
14 # /usr/bin/oocalc 
15 
LG 
Ly 


# Thank you, Rory Winston, for pointing this out. 
Lowercase conversion in parameter substitution (added in version 4 of Bash). 
escape [backslash]. A quoting mechanism for single characters. 


XX escapes the character X. This has the effect of "quoting" X, equivalent to 'X'. The \ may be used to 
quote " and ', so they are expressed literally. 


See Chapter 5 for an in-depth explanation of escaped characters. 


Filename path separator [forward slash]. Separates the components of a filename (as in 
/home/bozo/projects/Makefile). 


This is also the division arithmetic operator. 


command substitution. The ^command' construct makes available the output of command for 
assignment to a variable. This is also known as backquotes or backticks. 


null command [colon]. This is the shell equivalent of a "NOP" (no op, a do-nothing operation). It 


may be considered a synonym for the shell builtin true. The ":" command is itself a Bash builtin, and 
its exit status is true (0). 


aly yi 
2 echo $? # 0 


Endless loop: 


1 while : 


2 do 

3 operation-1 
4 operation-2 
5 em 

6 operation-n 
7 done 

8 

9 # Same as: 
10 4 while true 
11 # do 
12 # 


13 4 donor 
Placeholder in if/then test: 


JL xt iexguaxohi tium 

2i telaysin B 4 Do nothing and branch ahead 
3 else # Or else 

4 take-some-action 

5 seal 


Provide a placeholder where a binary operation is expected, see Example 8-2 and default parameters. 


1 : $(username-' whoami') 
2 # S(username-' whoami-~ } Gives an error without the leading 
3 d unless "username" is a command or builtin... 


Provide a placeholder where a command is expected in a here document. See Example 19-10. 
Evaluate string of variables using parameter substitution (as in Example 10-7). 


1: ${HOSTNAME?} ${USER?} ${MAIL?} 
2 # Prints error message 
3 #+ if one or more of essential environmental variables not set. 


Variable expansion / substring replacement. 


In combination with the > redirection operator, truncates a file to zero length, without changing its 
permissions. If the file did not previously exist, creates it. 


i > Gata. rom # File "data.xxx" now empty. 

2 

3 # Same effect as (Gene /clew/ mull scata es 

4 # However, this does not fork a new process, since ":" is a builtin. 


See also Example 16-15. 


In combination with the >> redirection operator, has no effect on a pre-existing target file (: >> 
target file).Ifthe file did not previously exist, creates it. 


cg; This applies to regular files, not pipes, symlinks, and certain special files. 


May be used to begin a comment line, although this is not recommended. Using # for a comment 
turns off error checking for the remainder of that line, so almost anything may appear in a comment. 
However, this is not the case with :. 


i § Thilg is cl Commence chart Gensrcarss an (xcxuouz, ( aie || Sx -SG 3] )) a 


The ":" also serves as a field separator, in /etc/passwd, and in the $PATH variable. 


bash$ echo $PATH 
/usr/local/bin:/bin:/usr/bin:/usr/Xl1R6/bin:/sbin:/usr/sbin:/usr/games 


reverse (or negate) the sense of a test or exit status [bang]. The ! operator inverts the exit status of 


the command to which it is applied (see Example 6-2). It also inverts the meaning of a test operator. 
This can, for example, change the sense of equal ( = ) to not-equal ( != ). The ! operator is a Bash 


keyword. 


In a different context, the ! also appears in indirect variable references. 


In yet another context, from the command line, the ! invokes the Bash history mechanism (see 
Appendix K). Note that within a script, the history mechanism is disabled. 


* 
wild card [asterisk]. The * character serves as a "wild card" for filename expansion in globbing. By 
itself, it matches every filename in a given directory. 
bash$ echo * 
abs-book.sgml add-drive.sh agram.sh alias.sh 
The * also represents any number (or zero) characters in a regular expression. 
x 
arithmetic operator. In the context of arithmetic operations, the * denotes multiplication. 
** A double asterisk can represent the exponentiation operator or extended file-match globbing. 
? 
test operator. Within certain expressions, the ? indicates a test for a condition. 
In a double-parentheses construct, the ? can serve as an element of a C-style trinary operator, ? :. 
l4 varc = verl<Soo79%2) 4) 
2 ^ ^ 
3 
4 ss | “SwaielY -le O8 I 
5 then 
6 var0-9 
7 else 
8 var0-21 
9 fea 
In a parameter substitution expression, the ? tests whether a variable has been set. 
? 
wild card. The ? character serves as a single-character "wild card" for filename expansion in 
globbing, as well as representing one character in an extended regular expression. 
$ 
Variable substitution (contents of a variable). 
1 vari-5 
2 var2-23skidoo 
3 
4 echo $varl 4 © 
5 echo $var2 # 23skidoo 
A $ prefixing a variable name indicates the value the variable holds. 
$ 
end-of-line. In a regular expression, a "$" addresses the end of a line of text. 
${} 


Parameter substitution. 
$*, $@ 
positional parameters. 


$? 


$$ 
0 


exit status variable. The $? variable holds the exit status of a command, a function, or of the script 
Itself. 


process ID variable. The $$ variable holds the process ID [2] of the script in which it appears. 
command group. 


1 (a-hello; echo $a) 
(D A listing of commands within parentheses starts a subshell. 


Variables inside parentheses, within the subshell, are not visible to the rest of the 
script. The parent process, the script, cannot read variables created in the child 
process, the subshell. 


a-123 
( a=321; ) 


ECCO NO ES 


echo "a = $a" # a = 123 
5 # "a" within parentheses acts like a local variable. 


array initialization. 


1 Array-(elementl1 element2 element3) 


{XXX,VVY,ZZZ,... } 


{a..z} 


u 


Brace expansion. 


echo \"{These, words, are, quoted} \" # " prefix and suffix 
# "These" "words" "are" "quoted" 


# Concatenates the files filel, file2, and file3 into combined_file. 


Cj i3.922 «1x5 loach} 
t Copies tii le22 esc! Ye) Mien e22. leveweliabyo ^ 


A command may act upon a comma-separated list of file specs within braces. [3] Filename 
expansion (globbing) applies to the file specs between the braces. 


1 
2 
3 
4 
5 (eue En lel, rnile2, dil] => Comlosineycl iene 
6 
E 
8 
3 


d No spaces allowed within the braces unless the spaces are quoted or escaped. 


echo {filel,file2}\ :{\ A," B",' C'} 


filel : A filel : B filel : C file2 : A file2 : B file2 
G 


Extended Brace expansion. 


echo (m.s) w em l9 c6 e e sims: 3 EE JL is G I9 Gre m BU YW ex Ww A 
Echoes characters between a and z. 


BwWN Ee 
+ 


echo q.s wu 9 3 2 3 
5 # Echoes characters between 0 and 3. 


The /a..z] extended brace expansion construction is a feature introduced in version 3 of Bash. 


Block of code [curly brackets]. Also referred to as an inline group, this construct, in effect, creates 
an anonymous function (a function without a name). However, unlike in a "standard" function, the 


variables inside a code block remain visible to the remainder of the script. 


bash$ { local a; 


a=123; } 
bash: local: can only be used ina 
function 
1 a=123 
2 { a=321; } 
3 echo Va = Sal # a = 321 (value inside code block) 
4 
5 a; "meus SoC. 


The code block enclosed in braces may have I/O redirected to and from it. 


Example 3-1. Code blocks and I/O redirection 


#!/bin/bash 
# Reading lines in /etc/fstab. 


File=/etc/fstab 


{ 

read linel 
read line2 
} < Smile 


acing "iubes Line im Swile age” 
echo "$linel" 

echo 

Sele Vecos ling im Grile abere V 
echo "$line2" 


exec O 


# Now, how do you parse the separate fields of each line? 
#2 kaime: quee ex. Ore 
# . . . Hans-Joerg Diers suggests using the "set" Bash builtin. 


(= 9 (Wer (Ger c ey (ab dE eS [ems [ 629 (Wer Csr cL rex; (Oa! gEx (93. [RS [5 


N N + 


Example 3-2. Saving the output of a code block to a file 


! /bin/bash 
rpm-check.sh 


Queries an rpm file for description, listing, 
+ and whether it can be installed. 
Saves output to a file. 


This script illustrates using a code block. 


SUCCESS=0 
E NOARGS-65 


ae (poc UBI] 

then 
echo "Usage: ~basename $0! rpm-file" 
exit SE NOARGS 

ftu 


sj «xy Gal dex (59) IS) [^ (Ex We) sh «| cxy (Gn) dex (Ge) Iss) IS 


H 


i15 


[] 


[ET] 


[] 


18 
19 ( # Begin code block. 


20 echo 

2L echo "Archive Description:" 

22) gan -ePi Sil # Query description. 

29) echo 

24 echo "Archive Listing:" 

25 wom -cpl Sil # Query listing. 

26 echo 

Z7 rpm -i --test $1 # Query whether rpm file can be installed. 
28 ii [ YSPY ee SSUCOCSESS ] 

2) then 

30 eao WSil can be ingrellec Y 

Si else 

22 echo Voil cannot oe imstalleci Y 

33 fra 

34 echo # End code block. 

Sg |} > Weiss # Redirects output of everything in block to file. 
36 

37 echo YReswWles qi cpm cest im isle El test” 

38 

39 # See rpm man page for explanation of options. 
40 

41 exit 0 


°) Unlike a command group within (parentheses), as above, a code block enclosed by 
(braces) will not normally launch a subshell. [4] 


placeholder for text. Used after xargs =i (replace strings option). The () double curly brackets are a 
placeholder for output text. 


Lis o | ses =i -i ep /() Sil 

2 # AN ^^ 

3 

4 # From "ex42.sh" (copydir.sh) example. 

anchor id="semicolonesc"> 

pathname. Mostly used in find constructs. This is not a shell builtin. 


g^) The ";" ends the -exec option of a find command sequence. It needs to be escaped to 
protect it from interpretation by the shell. 


test. 


Test expression between [ ]. Note that [ is part of the shell builtin test (and a synonym for it), not a 
link to the external command /usr/bin/test. 


test. 

Test expression between [[ ]]. More flexible than the single-bracket [ ] test, this is a shell keyword. 
See the discussion on the [[ ... |] construct. 

array element. 

In the context of an array, brackets set off the numbering of each element of that array. 


1 Array[1]-slot 1 


2 echo ${Array[1] } 


[] 


range of characters. 


As part of a regular expression, brackets delineate a range of characters to match. 
3|... ] 


integer expansion. 


Evaluate integer expression between §[ ]. 


echo $[Sa+tSb] # 10 
5 echo $[S$a*S$b] s DAL 


Note that this usage is deprecated, and has been replaced by the (( ... )) construct. 
(O) 


integer expansion. 
Expand and evaluate integer expression between (( )). 
See the discussion on the (( ... )) construct. 

> &> >& >> < <> 


redirection. 


scriptname >filename redirects the output of scriptname to file filename. Overwrite 
filename if it already exists. 


command &>filename redirects both the stdout and the stderr of command to filename. 


cg; This is useful for suppressing output when testing for a condition. For example, let us 
test whether a certain command exists. 


bash$ type bogus command &>/dev/null 
bash$ echo $? 
JL 

Or in a script: 


Commanclicesic () { tye "SI" &e/ckew/smllp p 
# ^ 


1 
2 
3 
4 cmd=rmdir # Legitimate command. 
5 comman test Senee echo $7 # 0 
6 
7 
8 
9 


cmd-bogus, command # Illegitimate command 
command_test $cmd; echo $? # 1 


command >&2 redirects stdout of command to stderr. 


scriptname >>filename appends the output of scriptname to file filename. If 
filename does not already exist, it is created. 


<< 


<<< 


\<, \> 


[i]<>filename opens file filename for reading and writing, and assigns file descriptor i to it. If 
filename does not exist, it is created. 


process substitution. 
(command) > 
< (command) 


In a different context, the "<" and ">" characters act as string comparison operators. 


In yet another context, the "<" and ">" characters act as integer comparison operators. See also 
Example 16-9. 


redirection used in a here document. 


redirection used in a here string. 


ASCII comparison. 


1 vegl=carrots 

2 veg2-tomatoes 

3 

4 ii [I "Swegl" < U"Sweg2" Ij 

5 then 

6 echo "Although $vegl precede $veg2 in the dictionary," 
7 echo -n "this does not necessarily imply anything " 

8 echo "about my culinary preferences." 

9 else 
JL) echo "What kind of dictionary are you using, anyhow?" 
IL Ei 


word boundary in a regular expression. 


bash$ grep '\<the\>' textfile 


pipe. Passes the output (stdout of a previous command to the input (stdin) of the next one, or to 
the shell. This is a method of chaining commands together. 


echo le —IJI | “sh 
# Passes the output of "echo ls -1" to the shell, 
tt with the same result as a simple "ls -1". 


Cue “ls | seme | waite 
# Merges and sorts all ".l1st" files, then deletes duplicate lines. 


cuj cx (Gal des (9 Je» qp 


A pipe, as a classic method of interprocess communication, sends the st dout of one process to the 
stdin of another. In a typical case, a command, such as cat or echo, pipes a stream of data to a 
filter, a command that transforms its input for processing. [5] 


cat $filenamel $filename2 | grep $search word 


For an interesting note on the complexity of using UNIX pipes, see the UNIX FAQ, Part 3. 


The output of a command or commands may be piped to a script. 


>l 


#!/bin/bash 
# uppercase.sh : Changes input to uppercase. 


il 
2 
3 
di eis Vieira Va 

5 # Letter ranges must be quoted 
6 

7 

8 


#+ to prevent filename generation from single-letter filenames. 


exit 0 
Now, let us pipe the output of Is -l to this script. 
bash$ ls -1 | ./uppercase.sh 


—RW-RW-R-— 1 BOZO BOZO LOS) mem y JX9849 I We 
BWR Ses 1 BOZO BOZO LOG wen al Weeds) 2 II 
PWR Ro 1 BOZO BOZO 725 APR 20 20:56 DATA-FILE 


ə The stdout of each process in a pipe must be read as the st din of the next. If this 
is not the case, the data stream will block, and the pipe will not behave as expected. 


i ceu tile is1e2 | ie =I | sext 
2 4 The output from "cat filel file2" disappears. 


A pipe runs as a child process, and therefore cannot alter script variables. 


1 variable-"initial value" 

2 echo "new value" | read variable 

3 echo "variable = $variable" # variable = initial, value 
If one of the commands in the pipe aborts, this prematurely terminates execution of the 
pipe. Called a broken pipe, this condition sends a SIGPIPE signal. 


force redirection (even if the noclobber option is set). This will forcibly overwrite an existing file. 


OR logical operator. In a test construct, the ll operator causes a return of 0 (success) if either of the 
linked test conditions is true. 


Run job in background. A command followed by an & will run in the background. 


bash$ sleep 10 & 
[3] 850 
[1]* Done Sleep 10 


Within a script, commands and even loops may run in the background. 


Example 3-3. Running a loop in the background 


#!/bin/bash 
# background-loop.sh 


for a xm dL 2S 45 6 y s 9 I0 # First loop. 
do 
exo =m "Sx wv 
done & # Run this loop in background. 
# Will sometimes execute after second loop. 


echo # This 'echo' sometimes will not display. 


for 2 im I 12 13 14 15 16 17 18 19 20 # Second loop. 
do 

Sena =a "Sn wv 
done 


PRPrPRPe PR 
OABRWNFPOWVOAIADOAWNE 


&& 


16 
Ly 
18 
19 
20 
2L 
22 
23 
24 
25 
26 
zu 
28 
29 
30 
Su 
22 
33 
34 
35 
36 
37 
38 
3g 
40 
41 
42 


echo # This 'echo' sometimes will not display. 


The expected output from the script: 
1234567 8 9 'iuidg 
ii 12 19 14 15 16 17 18 19 20 


Sometimes, though, you get: 

ii 12 13 14 215 16 17 19 39 20 

12345 67 8 S9 10 bozo S 

(The second 'echo' doesn't execute. Why?) 


Occasionally also: 
L123 45678 9 10 ii 22 13 14 15 16 17 18 419 20 
(The first 'echo' doesn't execute. Why?) 


Very rarely something like: 
di 12 19 3 2 94 5o yg 9 19 14 15 16 17 18 19 20 
The foreground loop preempts the background one. 


exit 0 

# Nasimuddin Ansari suggests adding sleep 1 
#+ after th cag =m ‘Sa, in lines 6 and 14, 
#+ for some real fun. 


A command run in the background within a script may cause the script to hang, 
waiting for a keystroke. Fortunately, there is a remedy for this. 


AND logical operator. In a test construct, the && operator causes a return of 0 (success) only if both 
the linked test conditions are true. 


option, prefix. Option flag for a command or filter. Prefix for an operator. Prefix for a default 
parameter in parameter substitution. 


COMMAND -[Option1] [Option2][...] 


ls -al 


sort -dfu $filename 


PRPPP PPP ee 
O0 -1 OY O! ;&. Q0. I9. IB. O «o OO -1 O OI iS CQ) IN S 


zac || Stilel ox Siedler | 
then # d 


echo "File $filel is older than $file2." 


fal 


IE [ TS ei -eq meg ] 


then 4 ii 
echo "$a is equal to $b." 
dE al 
if [ "Sc" -eq 24 -a "$d" -eq 47 ] 
then 4 e 2 


echo "Sc equals 24 and Sd equals 47." 


ial 


param2=S {paraml:-SDEFAULTVAL} 


# 


A 


The double-dash —— prefixes long (verbatim) options to commands. 
sort --ignore-leading-blanks 
Used with a Bash builtin, it means the end of options to that particular command. 


į ) This provides a handy means of removing files whose names begin with a dash. 


bash$ 1s -1 
=en JL lxewo loose 0 Woy 25 12929 coaches 


bash$ rm -- -badname 


bash$ ls =l 
total 0 


The double-dash is also used in conjunction with set. 
set —- $variable (as in Example 15-18) 
redirection from/to stdin or stdout [dash]. 


bash$ cat - 
abc 
abc 


Ctl-D 
As expected, cat — echoes stdin, in this case keyboarded user input, to stdout. But, does I/O 
redirection using - have real-world applications? 


i (eel /eourea/chirecrtory (e tar cur = 2 ) | (lec /cleSie/climecroimy ke tac govi =) 
2 Mov ntire file tr from one directory to another 
3 [courtesy Alan Cox <a.cox@swansea.ac.uk>, with a minor change] 
4 
5 1) cd /source/directory 
6 Source directory, where the files to be moved ar 
y 2) && 
8 UruselcdlstewEU8. ae tlie "eel" operar On GXEICIEKSSS HEU, 
9 then execute the next command. 
10 S tear Er = 
Lal The 'c' option 'tar' archiving command creates a new archive, 
12 the 'f' (file) option, followed by '-' designates the target fil 
13 as stdout, and do it in current directory tree ('.'). 
14 4 | 
L5) Piped to 
LG iow caca d) 
37 a subshell 
18 6) cd /dest/directory 
19 Change to the destination directory. 
20 7) && 
21 "And-list", as above 
p 8) ieee sowie = 
23 Unarchive ('x'), preserve ownership and file permissions ('p'), 
24 and send verbose messages to stdout ('v'), 
DID reading data from stdin ('f' followed by '-'). 
26 
27 Wore (Ele "sc" die. a Conmmamel, aincl anp Uy", YE" are GJOEIOms c 


28 4 


29 4 Whew! 

30 

SL 

32 

$19) More elegant than, but equivalent to: 

34 cd source/directory 

25 teur Qu — , | (eel ..JoesE/gohiecteryr tar sowie =) 
36 

3 Also having same effect: 

38 cp -a /source/directory/* /dest/directory 

3g (One e 

40 cp -a /source/directory/* /source/directory/.[^.]* /dest/directory 
41 If there are hidden files in /source/directory. 


i bunama =€ lamux=2.6. IG tai loz2 | tar seit = 

2 =—UNGOMOLESS ter ifile-- | e-tEmem pass it to Utsr-- 

3 If "tar" has not been patched to handle "bunzip2", 

4 #+ this needs to be done in two discrete steps, using a pipe. 

5 The purpose of th xercise is to unarchive "bzipped" kernel source. 


Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain 
UNIX utilities that write to stdout, such as tar, cat, etc. 


bash$ echo "whatever" | cat - 
whatever 


Where a filename is expected, - redirects output to stdout (sometimes seen with tar c£), or 
accepts input from st din, rather than from a file. This is a method of using a file-oriented utility as 
a filter in a pipe. 


bash$ file 
Usage: file [-bciknvzL] [-f namefile] [-m magicfiles] file... 


By itself on the command-line, file fails with an error message. 
Add a "-" for a more useful result. This causes the shell to await user input. 


bash$ file - 
abc 
standard input: ASCII text 


bash$ file - 
#!/bin/bash 
standard input: Bourne-Again shell script text executabl 


Now the command accepts input from st din and analyzes it. 


The "-" can be used to pipe stdout to other commands. This permits such stunts as prepending lines 
to a file. 


Using diff to compare a file with a section of another: 
grep Linux filel | diff file2 - 


Finally, a real-world example using — with tar. 


Example 3-4. Backup of all files changed in last day 


1 #!/bin/bash 
2 
3 Backs up all files in current directory modified within last 24 hours 
4 #+ in a "tarball" (tarred and gzipped file). 
5 
6 BACKUPFILE-backup-$ (date +%m-%d-%Y) 
7 Embeds date in backup filename. 
8 Thanks, Joshua Tschida, for the idea. 
9 archive-$(1:-$BACKUPFILE) 
10 If no backup-archive filename specified on command-line, 
JL3L spur Gig yall dkeftewsLi to "Uleyeveikqug— li - [DIDI YE CST SEA e pA a 
12 
ls car GWE = ame & me i -tyase t -prine S Seuselubwestus 
14 gzip $archive.tar 
l5 echo VDirescrtoty SEND backsc!i vwo in archiva tila earch yaota oy uU 
16 
UJ 
Le Stephane Chazelas points out that the above code will fail 
19 #+ if there are too many files found 
20 #+ or if any filenames contain blank characters. 
21 
2 He suggests the following alternatives: 
23 
24 ELME o meo il -cys dE -oriatt || zarga -0 tar rvi Variva, Teue” 
25 using the GNU version of "find". 
26 
27 
28 ELME o ED JL Sie E MOC re cyi WSereniyec teet Yi}! XE 
29 portable to other UNIX flavors, but much slower. 
30 
Sl 
32 
33 exit 0 


Filenames beginning with "-" may cause problems when coupled with the 
redirection operator. A script should check for this and add an appropriate prefix to 
such filenames, for example . / -FILENAME, SPWD/-FILENAME, or 

SPATHNAME /-FILENAME. 


If the value of a variable begins with a -, this may likewise create problems. 


JL yers =S ia 
2 echo Svar 
3 4 Has the effect of "echo -n", and outputs nothing. 


previous working directory. A cd - command changes to the previous working directory. This uses 
the SOLDPWD environmental variable. 


"non "non 


Do not confuse the "-" used in this sense with the "-" redirection operator just 
discussed. The interpretation of the "-" depends on the context in which it appears. 


Minus. Minus sign in an arithmetic operation. 


Equals. Assignment operator 
1 a-28 


2 eho fe # 28 
In a different context, the "=" is a string comparison operator. 


Plus. Addition arithmetic operator. 


% 


In a different context, the + is a Regular Expression operator. 
Option. Option flag for a command or filter. 


Certain commands and builtins use the + to enable certain options and the - to disable them. In 
parameter substitution, the + prefixes an alternate value that a variable expands to. 


modulo. Modulo (remainder of a division) arithmetic operation. 


(ete tz = Sis 3 
2 echo $z # 2 


In a different context, the % is a pattern matching operator. 


home directory [tilde]. This corresponds to the $HOME internal variable. ~bozo is bozo's home 
directory, and Is ~bozo lists the contents of it. ~/ is the current user's home directory, and Is ~/ lists the 
contents of it. 


bash$ echo ~bozo 
/home/bozo 


bash$ echo ~ 
/home/bozo 


bash$ echo ~/ 
/home/bozo/ 


bash$ echo ~: 
/home/bozo: 


bash$ echo -nonexistent-user 
^nonexistent-user 


current working directory. This corresponds to the $PWD internal variable. 

previous working directory. This corresponds to the $OLDPWD internal variable. 
regular expression match. This operator was introduced with version 3 of Bash. 
beginning-of-line. In a regular expression, a "^" addresses the beginning of a line of text. 


Uppercase conversion in parameter substitution (added in version 4 of Bash). 


Control Characters 


change the behavior of the terminal or text display. A control character is a CONTROL + key 
combination (pressed simultaneously). A control character may also be written in octal or 
hexadecimal notation, following an escape. 


Control characters are not normally useful inside a script. 
0 Ct1-A 


Moves cursor to beginning of line of text (on the command-line). 
0 Ct1-B 


Backspace (nondestructive). 
Y 
Ctl-C 


Break. Terminate a foreground job. 
0 
Ctl-D 


Log out from a shell (similar to exit). 
EOF (end-of-file). This also terminates input from st din. 


When typing text on the console or in an xterm window, Ct1-D erases the character under 
the cursor. When there are no characters present, CE 1-D logs out of the session, as expected. 
In an xterm window, this has the effect of closing the window. 

0 Ct1-E 


Moves cursor to end of line of text (on the command-line). 
0 ctl-F 


Moves cursor forward one character position (on the command-line). 
Y 
Ctl-G 


BEL. On some old-time teletype terminals, this would actually ring a bell. In an xterm it 
might beep. 

0 
Ctl-H 


Rubout (destructive backspace). Erases characters the cursor backs over while backspacing. 


But, this does not change the results. 


1 #!/bin/bash 
2 sp. lyiiloyerelolatiney Creil m sim a GENE 
3 
4 a-"^H^H" Two Ctl-H's -- backspaces 
E QEl-W qul-EBL dugsmep wi/ waa 
6 echo "abcdef" abcdef 
7 echo 
8 echo n "abcdefSa " abcd f 
9 # Space at end ^ ^  Backspaces twice. 
10 echo 
11 echo -n "abcdefSa" abcdef 
12 # No space at end ^ Doesn't backspace (why?). 
l3 Results may not be quite as expected. 
14 echo; echo 
L5 
16 # Constantin Hagemeier suggests trying: 
LD 4 eee NOIHONOSQT 
18 # a=S'\b\b' 
19 # a=$'\x08\x08' 
# 
I 


Horizontal tab. 
Ctl-J 
Newline (line feed). In a script, may also be expressed in octal notation -- ^012' or in 


hexadecimal -- ^xOa'. 
0 Ct1-K 


Vertical tab. 


When typing text on the console or in an xterm window, Ct1-K erases from the character 
under the cursor to end of line. Within a script, Ct1-K may behave differently, as in Lee Lee 
Maschmeyer's example, below. 

0 Ct1-L 


Formfeed (clear the terminal screen). In a terminal, this has the same effect as the clear 
command. When sent to a printer, a Ct 1—L causes an advance to end of the paper sheet. 


0 


Ctl-M 


Carriage return. 


1 #!/bin/bash 
2 4 Thank you, Lee Maschmeyer, for this example. 
3 
4 read -n 1 -s -p \ 
5 $'Control-M leaves cursor at beginning of this line. Press Enter. \x0d' 
6 # Of course, '0Od' is the hex equivalent of Control-M. 
7 echo >&2 # The '-s' makes anything typed silent, 
8 #+ so it is necessary to go to new line explicitly. 
9 
10 read -n 1 -s -p $'Control-J leaves cursor on next line. \x0a' 
dat #  '0a' is the hex equivalent of Control-J, linefeed. 
12 echo >&2 
13 
14 ### 
15 
le read -m 1 = =o SiNacl Comerol-k\z0bgoes Sicrangine dorm, v 
17 echo >&2 # Control-K is vertical tab. 
18 
19 # A better example of th ffect of a vertical tab is: 
20 
21 var-$'NxO0aThis is the bottom line\x0ObThis is the top line\x0a' 
22 echo "Svar" 
23 This works the same way as the abov xample. However: 
ZU ele) WSs || xexou 
25 This causes the right end of the line to be higher than the left end. 
26 It also explains why we started and ended with a line feed -- 
27 #+ to avoid a garbled screen. 
28 
29 As Lee Maschmeyer explains: 
30 
Sil In the [first vertical tab example] . . . the vertical tab 
32 #+ makes the printing go straight down without a carriage return. 
33 This is true only on devices, such as the Linux console, 
Sa aap iliac Cande CO Hoake V 
35 The real purpose of VT is to go straight UP, not down. 
36 It can be used to print superscripts on a printer. 
3 The col utility can be used to emulate the proper behavior of VT. 
3 
39 exit 0 
0 CE1-N 


Erases a line of text recalled from Aistory buffer [6] (on the command-line). 
0 Ct1-O 


Issues a newline (on the command-line). 
0 Ct1-P 


Recalls last command from history buffer (on the command-line). 
0 Ct1-Q 


Resume (XON). 


This resumes st din in a terminal. 
0 Ct1-R 


Backwards search for text in history buffer (on the command-line). 
0 Ct1-s 


Suspend (XOFF). 


This freezes stdin in a terminal. (Use Ctl-Q to restore input.) 
0 Ct1-T 


Reverses the position of the character the cursor is on with the previous character (on the 
command-line). 
0 Ct1-U 


Erase a line of input, from the cursor backward to beginning of line. In some settings, CC1-U 
erases the entire line of input, regardless of cursor position. 
0 Ct1-v 


When inputting text, CE1—V permits inserting control characters. For example, the following 
two are equivalent: 


i echo =a "Ws 
2 eeg Crell- Ne Cels 


Ct1-V is primarily useful from within a text editor. 
0 Ct1-W 


When typing text on the console or in an xterm window, Ct 1-W erases from the character 
under the cursor backwards to the first instance of whitespace. In some settings, CE1—W 
erases backwards to first non-alphanumeric character. 

0 Ct1-x 


In certain word processing programs, Cuts highlighted text and copies to clipboard. 
0 UCtleY 


Pastes back text previously erased (with Ct 1-U or CC 1-W). 
0 ct1-Z 


Pauses a foreground job. 
Substitute operation in certain word processing applications. 


EOF (end-of-file) character in the MSDOS filesystem. 
Whitespace 
functions as a separator between commands and/or variables. Whitespace consists of either 
spaces, tabs, blank lines, or any combination thereof. [7] In some contexts, such as variable 
assignment, whitespace is not permitted, and results in a syntax error. 


Blank lines have no effect on the action of a script, and are therefore useful for visually separating 
functional sections. 


SIES, the special variable separating fields of input to certain commands. It defaults to whitespace. 


Definition: A field is a discrete chunk of data expressed as a string of consecutive characters. 


Separating each field from adjacent fields is either whitespace or some other designated character 
(often determined by the $IFS). In some contexts, a field may be called a record. 


To preserve whitespace within a string or in a variable, use quoting. 
UNIX filters can target and operate on whitespace using the POSIX character class [:space:]. 
Notes 


[1] An operator is an agent that carries out an operation. Some examples are the common arithmetic 
operators, + - * /. In Bash, there is some overlap between the concepts of operator and keyword. 


2 
= A PID, or process ID, is a number assigned to a running process. The PIDs of running processes may 
be viewed with a ps command. 
Definition: A process is a currently executing command (or program), sometimes referred to as a 
job. 
[3] The shell does the brace expansion. The command itself acts upon the result of the expansion. 
[4] Exception: a code block in braces as part of a pipe may run as a subshell. 


ds | 4 zeac taresicilumes reac!) secol inep } 

# Error. The code block in braces runs as a subshell, 

#+ so the output of "Is" cannot be passed to variables within the block. 
echo Yrirst lins ius Sitdiestclimes secomel lime is Ssecomellime” d Wow i Tork. 


OX Cle GW NO ES 


w^ dUeunlsS SC 

[5] Even as in olden times a philtre denoted a potion alleged to have magical transformative powers, so 
does a UNIX filter transform its target in (roughly) analogous fashion. (The coder who comes up with a 
"love philtre" that runs on a Linux machine will likely win accolades and honors.) 


[6] Bash stores a list of commands previously issued from the command-line in a buffer, or memory space, 
for recall with the builtin history commands. 


[7] A linefeed (newline) is also a whitespace character. This explains why a blank line, consisting only of a 
linefeed, is considered whitespace. 
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Chapter 4. Introduction to Variables and 
Parameters 


Variables are how programming and scripting languages represent data. A variable is nothing more than a 
label, a name assigned to a location or set of locations in computer memory holding an item of data. 


Variables appear in arithmetic operations and manipulation of quantities, and in string parsing. 


4.1. Variable Substitution 


The name of a variable is a placeholder for its value, the data it holds. Referencing (retrieving) its value is 
called variable substitution. 


$ 


Let us carefully distinguish between the name of a variable and its value. If variablel is the name 
of a variable, then $variablel is a reference to its value, the data item it contains. [1] 


bash$ variablel-23 


bash$ echo variablel 
variablel 


bash$ echo $variablel 
23 


The only time a variable appears "naked" -- without the $ prefix -- is when declared or assigned, when 
unset, when exported, or in the special case of a variable representing a signal (see Example 31-5). 
Assignment may be with an = (as in var 1-27), in a read statement, and at the head of a loop (for 
var2 in 1 2 3) 


Enclosing a referenced value in double quotes (" ... ") does not interfere with variable substitution. 
This is called partial quoting, sometimes referred to as "weak quoting." Using single quotes (' ... ) 
causes the variable name to be used literally, and no substitution will take place. This is full quoting, 
sometimes referred to as 'strong quoting.' See Chapter 5 for a detailed discussion. 


Note that $variable is actually a simplified form of $ {variable}. In contexts where the 
$variable syntax causes an error, the longer form may work (see Section 10.2, below). 


Example 4-1. Variable assignment and substitution 


#!/bin/bash 
# ex9.sh 


# Variables: assignment and substitution 


a=375 
hello-$a 


What happens if there is a space? 


"VARIABLE -value" 


^ 


oe 


Script tries to run "VARIABLE" command with one argument, "-value". 


"VARIABLE- value" 


^ 


1 
2 
3 
4 
5 
6 
Ji 
8 
9 
10 No space permitted on either side of - sign when initializing variables. 
al 
2 
3 
4 
5 
6 
7 
8 


Ne} 
oe 


Script tries to run "value" command with 
the environmental variable "VARIABLE" set to "". 


N 
ce 
ES 


24 echo hello # hello 
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Not a variable reference, just the string "hello" 


echo $hello # 375 

e This *is* a variable reference. 
exce STeello) + 375 

Also a variable reference, as above. 


Quoting s 
echo "Shello" 7 StS 
emo "Simelloj" 4 3795 
echo 
hello-"A B C DJ 
echo shell # ABCD 
cie; “Sianeli! ci € D 
As you see, echo Shello and eho "eme" giv 
Why? 
Quoting a variable preserves whitespac 
echo 
echo '$hello' # $hello 
Variable referencing disabled (escaped) by single quotes, 


+ which causes the "$" to be interpreted literally. 


Notice the effect of different types of quoting. 


hello- # Setting it to a null value. 
exo "XSmello (all ele) = S$mello" 


ghuitirteueewoum te 


* unsetting it, although the end result is the sam 


(s 


Note that setting a variable to a null value is not the same as 
below). 


+ if separated by white space. 


varl=21  var2-22  var3-$V3 
echo 
echo "varl-$varl var2-$var2 var3=Svar3" 


# May cause problems with older versions of "sh" 


# 


It is permissible to set multiple variables on the same line, 


Caution, this may reduce legibility, and may not be portable. 


echo; echo 


numbers="one two three" 


other_numbers="1 2 3" 
If there is whitespac mbedded within a variable, 
+ then quotes are necessary. 
other_numbers=1 2 3 # Gives an error messag 
echo "numbers = $numbers" 
echo "other numbers = Sother_numbers" # other numbers = 1 2 3 
Escaping the whitespace also works. 
mixed_bag=2\ \ Whatever 
2 ^ Space after escape (V). 


suia Ss 


Ong cchom mue cat o # 2 --- Whatever 
92 
93 echo; echo 
94 
95 echo "uninitialized variable = Suninitialized_variable" 
96 # Uninitialized variable has null value (no value at all!). 
97 uninitialized_variable= 7 BDoclearinep louc MOE dmieied zike aie == 
98 #+ same as setting it to a null value, as above. 
99 echo "uninitialized variable = $uninitialized variable" 
100 s; dae SELO bas a null welwe, 
101 
102 uninitialized variable-23 i; OGE Bie. 
103 unset uninitialized_variable # Unset it. 
104 echo "uninitialized variable = $uninitialized variable" 
105 i dee Stili has a mI welines, 
106 echo 
107 
108 exit 0 


® 


An uninitialized variable has a "null" value -- no assigned value at all (not zero!). 


d. if [| ex "Stinassieinecl” | 

2, ELEn 

3 echo "\Sunassigned is NULL." 

A iE3L 4 Sunassigned is NULL. 
Using a variable before assigning a value to it may cause problems. It is nevertheless 
possible to perform arithmetic operations on an uninitialized variable. 


JL eio YW Swine well abere # (blank line) 
2 let "uninitialized += 5" # Add 5 to it. 
3 echo "Suninitialized" hit) us) 

4 

5 (Cioxeve JL 853. ra. € 

6 An uninitialized variable has no value, 

7 #+ however it acts as if it were 0 in an arithmetic operation. 

8 This is undocumented (and probably non-portable) behavior, 

9 #+ and should not be used in a script. 


See also Example 15-23. 
Notes 


[1] Technically, the name of a variable is called an lvalue, meaning that it appears on the left side of an 
assignment statment, as in VARIABLE-23. A variable's value is an rvalue, meaning that it appears on 
the right side of an assignment statement, as in VAR2=$VARIABLE. 


A variable's name is, in fact, a reference, a pointer to the memory location(s) where the actual data 
associated with that variable is kept. 
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4.2. Variable Assignment 


the assignment operator (no space before and after) 


d Do not confuse this with = and -eq, which test, rather than assign! 


Note that = can be either an assignment or a test operator, depending on context. 


Example 4-2. Plain Variable Assignment 


#!/bin/bash 
# Naked variables 


echo 


# When is a variable "naked", i.e., lacking the '$' in front? 
# When it is being assigned, rather than referenced. 


# Assignment 
a=879 
eche “tine valve or \Wa\" xg Sm" 


# Assignment using 'let' 
let B= Ios 


echo Vihe valve of Yat as mow Sa,” 


echo 


# Ina 'for' loop (really, a type of disguised assignment): 
echo — MWeallties: gue wc aie cne loos gue UU 
tor a xm 7 @ 9 Xil 
do 
echo =n "Sa X 
done 


[3 RS [RS [RS [S iS ee eee gr gt 
iS) [sy A c9» (er (Ger p ey (up des (EX. I [| ©) (Wer Cs) c3 rex) (Gn dEx (os) I) [5 


N N 
ow 


echo 
echo 


NO NM 
-] O0 


N 
[99] 


# In a 'read' statement (also a type of assignment): 
echo =i "igienEgue WEN UU 

read a 

echo "The value of \"a\" is now $a." 


echo 


C9 CO CO CO CO CO CO DN 
(exy (Gn dex (eX. [ey [p ce». del 


esac 0) 


Example 4-3. Variable Assignment, plain and fancy 


#!/bin/bash 


a=23 # Simple case 
echo Sa 

b=Sa 

echo $b 


sa] ey) (Gn) PSS (es) [eS [55 


8 # Now, getting a little bit fancier (command substitution). 


9 
10 a= echo Hello!" # Assigns result of 'echo' command to 'a' 
11 echo $a 
12 # Note that including an exclamation mark (!) within a 


13 #+ command substitution construct will not work from the command-line, 
14 #+ since this triggers the Bash "history mechanism." 

15 # Inside a script, however, the history functions are disabled. 

16 


17 a= ls -1l^ # Assigns result of 'ls -1' command to 'a' 

18 echo $a # Unquoted, however, it removes tabs and newlines. 
19 echo 

AQ gehio "Sg" # The quoted variable preserves whitespac 

EN # (See the chapter on "Quoting.") 

22 

23 exit 0 


Variable assignment using the $(...) mechanism (a newer method than backquotes). This is actually a 
form of command substitution. 


JL $$ pron Jesse s losal 
2 R=S(cat /etc/redhat-release) 
3 arch-$(uname -m) 
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4.3. Bash Variables Are Untyped 


Unlike many other programming languages, Bash does not segregate its variables by "type." Essentially, Bash 
variables are character strings, but, depending on context, Bash permits arithmetic operations and 
comparisons on variables. The determining factor is whether the value of a variable contains only digits. 


Example 4-4. Integer or string? 


1 #!/bin/bash 

2, ip IMME Cie Sie alias), Sila 

3 

4 a-2334 Integer. 

5 let "a += 1" 

6 echo "a = $a " a = 2335 

7 echo Integer, still. 

8 

9 

10 b-$(a/23/BB) SuioeExtute Wi" ier UOS. 

Li WAALS Exeewowedtowcwns SO NED a SEXUS 

12 echo "b = $b" b = BB35 

13 declare -i b Declaring it an integer doesn't help. 
14 echo "b = $b" b = BB35 

15 

Jio dete, Wie: ar pU BIBS s di 

17 echo "b = $b" b-1 

18 echo Bash sets the "integer value" of a string to 0. 
19 
20 c=BB34 
Zi Selio Ve es. Seu c = BB34 
22 gieS4e/18/23] Subst rette "23 seis Ug. 
259 This makes $d an integer. 
24 echo "d = $d" d = 2334 
25 dec, "eb sp pU BISA s il 
26 echo "d = $d" Gl 25395 
27 echo 
28 
29 
30 4 What about null variables? 
Saee usos. On 
32 echo "e - $e" e- 
33 let "e += 1" Arithmetic operations allowed on a null variable? 
34 echo "e = $e" e-1 
35 echo Null variable transformed into an integer. 
36 
37 4 What about undeclared variables? 
S8eecho. MED ES £f = 
SO OE SPEECH Arithmetic operations allowed? 

419) «xcloyg) "ig = Sup f= 1 

41 echo Undeclared variable transformed into an integer. 
42 

43 However 

44 let "f /= Sundecl_var" Divide by zero? 

45 deis i /S 3 syntar orrors (pex Seral (arron Tokan sg W w) 
46 Syntax error! Variable $undecl_var is not set to zero here! 

47 

48 Buie. eeii 

49 let "f /= 0" 

Soi Weis 3E f= 8 Ceen a Tox. (0) (Screws ieee ae Uu 

Sil Expected behavior. 

52 


Oo 
w 


54 Bash (usually) sets the "integer value" of null to zero 
55 #+ when performing an arithmetic operation. 

56 But, Conie Emy cais du meme, folks! 

57 It's undocumented and probably non-portable behavior. 
5E 

59 

60 Conclusion: Variables in Bash are untyped, 

61 #+ with all attendant consequences. 

62 

63 ezit SP 


Untyped variables are both a blessing and a curse. They permit more flexibility in scripting and make it easier 
to grind out lines of code (and give you enough rope to hang yourself!). However, they likewise permit subtle 
errors to creep in and encourage sloppy programming habits. 


To lighten the burden of keeping track of variable types in a script, Bash does permit declaring variables. 
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4.4. Special Variable Types 


Local variables 

Variables visible only within a code block or function (see also local variables in functions) 
Environmental variables 

Variables that affect the behavior of the shell and user interface 


8^; In a more general context, each process has an "environment", that is, a group of 
variables that the process may reference. In this sense, the shell behaves like any other 
process. 


Every time a shell starts, it creates shell variables that correspond to its own 
environmental variables. Updating or adding new environmental variables causes the 
shell to update its environment, and all the shell's child processes (the commands it 
executes) inherit this environment. 


The space allotted to the environment is limited. Creating too many environmental 
variables or ones that use up excessive space may cause problems. 


bash$ eval "seq 10000 | sed -e 's/.*/export var&-ZZZZZZZZZZZZZZ/"''" 


bash$ du 
bash: /usr/bin/du: Argument list too long 


Note: this "error" has been fixed, as of kernel version 2.6.23. 


(Thank you, Stéphane Chazelas for the clarification, and for providing the above 
example.) 
If a script sets environmental variables, they need to be "exported," that is, reported to the 
environment local to the script. This is the function of the export command. 


$^) A script can export variables only to child processes, that is, only to commands or 
processes which that particular script initiates. A script invoked from the 
command-line cannot export variables back to the command-line environment. 
Child processes cannot export variables back to the parent processes that spawned 
them. 


Definition: A child process is a subprocess launched by another process, its 


parent. 
Positional parameters 
Arguments passed to the script from the command line [1] : $0, $1, $2, $3... 


$0 1s the name of the script itself, $1 is the first argument, $2 the second, $3 the third, and so forth. 
[2] After $9, the arguments must be enclosed in brackets, for example, $(10), ${11},${12}. 


The special variables $* and $@ denote all the positional parameters. 


Example 4-5. Positional Parameters 


1 #!/bin/bash 

2 

3 # Call this script with at least 10 parameters, for example 
4 4 ./scriptname 123 45 67 8 9 10 


5 MINPARAMS-10 
6 
7 echo 
8 
9) echo "ume Meme Qut tnis serio as WU. 
10 # Adds ./ for current directory 
11 echo "The name of this script is Ni basename $0°\"." 
12 # Strips out path name info (see 'basename') 
13 
14 echo 
15 
LWE aic | =m "S1" |] # Tested variable is quoted. 
17 then 
18 echo "Parameter #1 is $1" # Need quotes to escape # 
US su 
20 
AL aise [| — USO | 
22 then 
23 eleg; VWiPeicennsicence y2 io Sew 
ZA tas 
25 
26 ae || em YSU || 
27 then 
28 echo "Parameter #3 is $3" 
AS) we aL 
30 
31 # 
32 
33 
34 if [ -n "${10}" ] # Parameters > $9 must be enclosed in {brackets}. 
35 then 
36 echea YParcemerer iO is S410) 
Sy Ei 
38 
39) ehg V M 
40 echo "All the command-line parameters are: "$*"" 
41 
42 if [ $4 -lt "SMINPARAMS" ] 
43 then 
44 echo 
45 echo "This script needs at least SMINPARAMS command-line arguments!" 
AG iu 
47 
48 echo 
49 
50 exit 0 


Bracket notation for positional parameters leads to a fairly simple way of referencing the last 
argument passed to a script on the command-line. This also requires indirect referencing. 


args=S# # Number of args passed. 
lastarg=S{!args} 
# Note: This is an *indirect reference* to $args 


# Or: lastarg=${!#} (Thanks, Chris Monson.) 
# This is an *indirect reference* to the $# variable. 


1 
2 
3 
4 
5 
6 
7 
8 # Note that lastarg=${!S#} doesn't work. 

Some scripts can perform different operations, depending on which name they are invoked with. For 
this to work, the script needs to check $0, the name it was invoked by. There must also exist symbolic 


links to all the alternate names of the script. See Example 16-2. 


į ) If a script expects a command-line parameter but is invoked without one, this may 
cause a null variable assignment, generally an undesirable result. One way to prevent 
this is to append an extra character to both sides of the assignment statement using the 
expected positional parameter. 


variablel_=S1_ # Rather than variablel-$1 
# This will prevent an ven if positional parameter is absent. 


acie ue", 
critical argumentO0l1-$variablel . 
The extra character can be stripped off later, like so. 
wieucaedole1-sSwsuazlsled / /) 

ide effects only if Svariablel_ begins with an underscore. 


S 
This uses one of the parameter substitution templates discussed later. 
(Leaving out the replacement pattern results in a deletion.) 


A more straightforward way of dealing with this is 


* to simply test whether expected positional parameters have been passed. 
we [p oed | 
then 
exit $E MISSING POS PARAM 
ít 


NPRPRPPRP PPP PY 
O (o 0» -1 O Oi i (). M I2. O «o 0 -1 O O1 4 CQ I9 E 


# However, as Fabian Kreutz points out, 
the above method may have unexpected sid 
A better method is parameter substitution: 
${1:-SDefaultVal } 

"Parameter Substition" section 
"Variables Revisited" chapter. 


PEECES: 


NNN PN 
iw GON ES 
Se HE OE 


See the 
in the 


N 
U 


Example 4-6. wh, whois domain name lookup 


!/bin/bash 

ex18.sh 

Does a 'whois domain-name' lookup on any of 3 alternate servers: 
ripe.net, cw.net, radb.net 

Place this script -- renamed 'wh' -- in /usr/local/bin 


Requires symbolic links: 

ln -s /usr/local/bin/wh /usr/local/bin/wh-ripe 
ln -s /usr/local/bin/wh /usr/local/bin/wh-apnic 
ln -s /usr/local/bin/wh /usr/local/bin/wh-tucows 


E NOARGS-75 


af 

then 
echo "Usage: ~basename $0" 
exit $E NOARGS 

ít 


ug ] 


m 


[domain-name]" 


Mop pop pp PPP PY 
O (o 0» -1 O Oi i5 (). NN PS. O xo 0 -1 O O1 4 QI 


NNN 
w Ne 


# Check script name and call proper server. 


24 case `basename $0` in # Or: case ${0##*/} in 

25 "wh" ) whois $1@whois.tucows.com;; 

26 "wh-ripe" ) whois $1@whois.ripe.net;; 

27 "wh-apnic" ) whois $1@whois.apnic.net;; 

28 "wh-cw" ) whois $1@whois.cw.net;; 

29 5s ) echo "Usage: ^basename $0° [domain-name]";; 


30 esac 
Sil 
32 exit S7 


The shift command reassigns the positional parameters, in effect shifting them to the left one notch. 
$1 <--- $2, $2 <--- $3, $3 <--- $4, etc. 
The old $1 disappears, but $0 (the script name) does not change. If you use a large number of 


positional parameters to a script, shift lets you access those past 10, although {bracket} notation also 
permits this. 


Example 4-7. Using shift 


#!/bin/bash 
# shft.sh: Using 'shift' to step through all the positional parameters. 


# Name this script something like shft.sh, 
#+ and invoke it with some parameters. 
#+ For example: 


# Sin Sluice. sin m lo e cer 25 Skidoo 
tinea [D ee VSL" | s meil all parameters USEC us 
do 

eco =a WS w 

shift 
done 
echo # Extra linefeed. 


# But, what happens to the "used-up" parameters? 

exo "SW 

# Nothing echoes! 

i nem $2 seite inco Sil (ewe utere ig mo 93 to shiit imtoo 92) 


Mop pop pp PrP PP 
O io 00 -1 O Oi i$ (0. NM. IS. O xo 0 -1 O O1 i& CQ) IO ES 


21 #+ then $2 remains empty. 

22 # So, it is not a parameter *copy*, but a *move*. 

23 

24 exit 

25 

26 # See also the echo-params.sh script for a "shiftless" 

27 #+ alternative method of stepping through the positional params. 


The shift command can take a numerical parameter indicating how many positions to shift. 


#!/bin/bash 
# shift-past.sh 


emit 3 i? Smrke 3 ISO: 9m. 
5$ je sip heir Sm 
# Has the same effect. 


echo urnim 


exit 0 


PRR 
WNHROOMIDADAAOBRWBNE 


14 
15 $ sh suiit-pest.sm 2 34 5 


16 4 

L7 

L8 However, as Eleni Fragkiadaki, points out, 

19 #+ attempting a 'shift' past the number of 

20 #+ positional parameters ($4) returns an exit status of 1, 
21 #+ and the positional parameters themselves do not change. 
22 This means possibly getting stuck in an endless loop. 
23 For example: 

24 UME [ —2 Vei | 

25 do 

26 echo m Wisil Ww 

27 Gimme 20) # If less than 20 pos params, 

28 done #+ then loop never ends! 

29 

30 When in doubt, add a sanity check. 

3 shire 29) || esses 

32 durae aue 


$^) The shift command works in a similar fashion on parameters passed to a function. See 
Example 35-16. 


Notes 


[1] Note that functions also take positional parameters. 


[2] The process calling the script sets the $0 parameter. By convention, this parameter is the name of the 
script. See the manpage (manual page) for execv. 


From the command-line, however, $0 1s the name of the shell. 


bash$ echo $0 
bash 


tcsh$ echo $0 
tcsh 
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Chapter 5. Quoting 


Quoting means just that, bracketing a string in quotes. This has the effect of protecting special characters in 
the string from reinterpretation or expansion by the shell or shell script. (A character is "special" if it has an 
interpretation other than its literal meaning. For example, the asterisk * represents a wild card character in 


globbing and Regular Expressions). 


bash$ ls -1 [Vv]* 


Saye 1 — JL Isxorre) D070 324 Apr 2 15:05 VIEWDATA.BAT 
CEW EW E iL bozo DOZO 507 May 4 14:25 vartrace.sh 
TEW AN G == L bozo bozo 539 Apr 14 17:11 viewdata.sh 


bash$ ls -1 '[Vv]*' 
less [Wiww]*s No cuch rila gis Gbusectonry 


In everyday speech or writing, when we "quote" a phrase, we set it apart and give it special meaning. In a 
Bash script, when we quote a string, we set it apart and protect its literal meaning. 


Certain programs and utilities reinterpret or expand special characters in a quoted string. An important use of 
quoting is protecting a command-line parameter from the shell, but still letting the calling program expand it. 


bash$ grep '[Ff]lirst' *.txt 
iride jedes abes clove jealiesic Iain (ur allel VENUES 
iade2.:s5zeuns 1S iue Parat line OE isle! ENSE 


Note that the unquoted grep [Ff]irst *.txt works under the Bash shell. [1] 
Quoting can also suppress echo's "appetite" for newlines. 


bash$ echo $(1s -1) 
Total B —Euw-zwuw-i:-— I loo leo 13 Aug 21 12857 t.Sin —Ewr-xwgy-i-— dL. loo loo 739 Aug 21 12957 w.sm 


bash$ echo "$(1s -1)" 

totale 

Sawa 1 leo loo 13 Aug 2X 12857 Tosi 
—ugEwy—sew-i:s—— 3L loo loo 79 Ate AL 12857 ws 


5.1. Quoting Variables 


When referencing a variable, it is generally advisable to enclose its name in double quotes. This prevents 
reinterpretation of all special characters within the quoted string -- except $, ` (backquote), and \ (escape). [2] 
Keeping $ as a special character within double quotes permits referencing a quoted variable 
("Svariable"), that is, replacing the variable with its value (see Example 4-1, above). 


Use double quotes to prevent word splitting. [3] An argument enclosed in double quotes presents itself as a 
single word, even if it contains whitespace separators. 


List-"one two three" 


fe» @ in Sus Sie # Splits the variable in parts at whitespace. 
do 
COoN S 
done 
# one 
# two 
# three 


exem) Wao! 


ror em im VELLE # Preserves whitespace in a single variable. 
do # S 
echo "Sg" 
16 done 
17 # one two three 


A more elaborate example: 


(Ont dE (G3) [S3) [5 6S) Wey Csi c cx) (rl des (€ I A 


variablel-"a variable containing five words" 
COMMAND This is $variablel # Executes COMMAND with 7 arguments: 
i? agmine" Vag eU Uveneitelole'! Veemecalimalioe! "sex Uses 


COMMAND "This is $variablel" # Executes COMMAND with 1 argument: 
# "This is a variable containing five words" 


variable2="" Empty. 


COMMAND $variable2 $variable2 Svariable2 

Executes COMMAND with no arguments. 

COMMAND "$variable2" "Svariable2" "Svariable2" 

Executes COMMAND with 3 empty arguments. 
COMMAND "$variable2 $variable2 $variable2" 

Executes COMMAND with 1 argument (2 spaces). 


=] cxy (Gal de» Go) IS) [5 SS) (We) Ce | ny (Gn) de (9 ISS) [S 


[09] 
E 
H 


hanks, Stéphane Chazelas. 


į ) Enclosing the arguments to an echo statement in double quotes is necessary only when word splitting or 
preservation of whitespace is an issue. 


Example 5-1. Echoing Weird Variables 


#!/bin/bash 
# weirdvars.sh: Echoing weird variables. 


echo 


(ni ges (8) ISS) ps 


6 suec" (ALAE E 

7 echo $var ur Y NCS 

8 acho “Siweie! # '(J\{}$" Dessm tt male a cCliriereimes , 

B 

10 echo 

LL 

12 3ngee UN v 

JL3 ceho Svar ue Ut) SA \ converted to space. Why? 
IA echo Veveri # GAT Ew 

LS 

16 # Examples above supplied by Stephane Chazelas. 
ly 

18 echo 

Lg 
210) sg ZEN SNO SOSA 
21 echo $var2 # Ui 
BE echo Veven? # \\" 
23 echo 
QUAL Giz BUE gas VenA NNNNUU. a alegea Mie 
25 eure A 
20 echo "Sirene sul # \\\\ 
27 # Strong quoting works, though. 
28 
29 exit 


Single quotes ('') operate similarly to double quotes, but do not permit referencing variables, since the special 
meaning of $ is turned off. Within single quotes, every special character except ' gets interpreted literally. 
Consider single quotes ("full quoting") to be a stricter method of quoting than double quotes ("partial 


quoting"). 


H) Since even the escape character (V) gets a literal interpretation within single quotes, trying to enclose a 


single quote within single quotes will not yield the expected result. 


1 echo "Why can't I write 's between single quotes" 


2 

3 echo 

4 

5 # The roundabout method. 

6 echo 'Why can'\''t I write '"'"'s between single quotes' 

7 d | | | | | 

8 # Thr single-quoted strings, with escaped and quoted singl 

E 

10 4 This example courtesy of Stéphane Chazelas. 
Notes 


quotes betw 


[1] Unless there is a file named first in the current working directory. Yet another reason to quote. 


(Thank you, Harald Koenig, for pointing this out. 
2I 


myn 


Encapsulating 


within double quotes gives an error when used from the command line. This is 


interpreted as a history command. Within a script, though, this problem does not occur, since the Bash 


history mechanism is disabled then. 


Of more concern is the apparently inconsistent behavior of \ within double quotes, and especially 


following an echo -e command. 


bash$ echo hello\! 
hello! 

bash$ echo "hello\!" 
jae 1e ii 


bash$ echo "VN" 
bash$ echo Na 


bash$ echo "Na" 


bash$ echo xNty 


bash$ echo "x\ty" 


bash$ echo -e x\ty 


xty 
bash$ echo -e "x\ty" 
x 4 


Double quotes following an echo sometimes escape X. Moreover, the —e option to echo causes the "Xt" 
to be interpreted as a tab. 


(Thank you, Wayne Pollock, for pointing this out, and Geoff Lee and Daniel Barclay for explaining it.) 


[3] "Word splitting," in this context, means dividing a character string into separate and discrete arguments. 
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5.2. Escaping 


Escaping is a method of quoting single characters. The escape (\) preceding a character tells the shell to 
interpret that character literally. 


With certain commands and utilities, such as echo and sed, escaping a character may have the opposite 
effect - it can toggle on a special meaning for that character. 


Special meanings of certain escaped characters 


used with echo and sed 


\n 

means newline 
\r 

means return 
\t 

means tab 
\v 

means vertical tab 
\b 

means backspace 
\a 

means alert (beep or flash) 
\Oxx 


translates to the octal ASCII equivalent of Onn, where nn is a string of digits 


Example 5-2. Escaped Characters 


#!/bin/bash 
# escaped.sh: escaped characters 


echo; echo 


# Escaping a newline. 


echo WW 


echo "This will print 
as two lines." 

# This will print 

# as two lines. 


exeleg. Wallis wali ozige N 
as one line." 
# This will print as one line. 


Re [m3 [RR Mi SP eS eS eee ees 
WV A c9» (Wer (Gar — ey) (ab PS (eX) I [| 69» (Wer sl c ep) (Gn Ex (es) IS) [5 


cho "============= ut 
24 
2:3. eel ww ww! # Prints \v\v\v\v literally. 
26 # Use the -e option with 'echo' to print escaped characters. 
27 echo "l-———————---— i 
28 echo "VERTICAL TABS" 
29 echo -e "\v\v\v\v" # Prints 4 vertical tabs. 
30 echo " " 


31l 
32 echo "QUOTATION MARK" 


33 eelag e VY\ OAD # Prints " (quote, octal ASCII character 42). 
Sl echo Y y 
35 


36 # The $'\X' construct makes the -e option unnecessary. 
37 echo; echo "NEWLINE AND BEEP" 


38 echo (Si Wm # Newline. 

39) eeno SY We # Alert (beep). 
40 

41 echo " " 


42 echo "QUOTATION MARKS" 


43 Version 2 and later of Bash permits using the $'\nnn' construct. 
44 Note tamet ain tnla Case, V\awain! as am occa velte, 

45 echo $'\t \042 Nt' # Quote (") framed by tabs. 

46 

47 It also works with hexadecimal values, in an $'\xhhh' construct. 
4$ echo SW VEZ Wr" 47 OuceS (Y) framad oy tabs. 

49 Thank you, Greg Keraunen, for pointing this out. 

50 Earlier Bash versions allowed '\x022'. 

Bil Geno U " 

52 echo 

53 

54 

55 

56 

5/7 

58 # Assigning ASCII characters to a variable. 

59 # 

60 quote-$'V042' # " assigned to a variable. 


61 echo "$quote This is a quoted string, $quote and this lies outside the quotes." 
62 

63 echo 

64 

65 4 Concatenating ASCII chars in a variable. 

GG triple wuevoleuelbiu eS V VI ST VIST VISTI! a 137 is octal ASCII cole itor " V 
67 echo "Striple_underline UNDERLINE Striple_underline" 

68 

69 echo 

70 

TEASER NOLO ur Oil, 102, IOS are octal A, B, Cs. 
72 echo SABC 

73 

74 echo; echo 

US 

76 escape=$'\033' # 033 is octal for escape. 

77 echo "\"escape\" echoes as Sescape" 

78 # no visible output. 

79 

80 echo; echo 

81 

82 exit 0 


See Example 36-1 for another example of the $' ... ' string-expansion construct. 
gives the quote its literal meaning 


i echo "sello" # Hello 
2 Gemo UN USED. 255 lave: Seleh Y s Wig xs he Geel, 


gives the dollar sign its literal meaning (variable name following V$ will not be referenced) 


1 echo "\Svariable01" # SvariableO1 
2 eche “The book cost \S7.98." s Tne book cost 975.98. 


\\ 
gives the backslash its literal meaning 


1 echo "NN" 4 Results in \ 

2 

3 # Whereas 

4 

5 echo WW # Invokes secondary prompt from the command-line. 
6 # In a script, gives an error messag 

7 

8 # However 

9 

i9 echo "Yt # Results in \ 


$^) The behavior of \ depends on whether it is escaped, strong-quoted, weak-quoted, or appearing within 


command substitution or a here document. 


1 Simple escaping and quoting 
2 echo Wm Z 
3 echo WW Nz 
4 echo "NU Nz 
5 echo "NW? We 
6 echo UN yz 
T cela) WYN Nz 
8 
9 Command substitution 
10 echo ‘echo \z` Z 
iid echo “echo Yz Z 
12 echo echo NNNz^ NR 
i3 echo echo YE NZ 
I4 echo echo: Annaa D: 
15 echo echo WW NN WES We 
16 exelave; echo Vee) \Z 
iy eels n echo NANa XN 
18 
19 Here document 
A) cart EIOS 
2l yz 
22 BOT NZ 
23 
2 eat seems 
25 XXe 
26 EOF 4 \z 
2 


28 4 These examples supplied by Stéphane Chazelas. 
Elements of a string assigned to a variable may be escaped, but the escape character alone may not be 
assigned to a variable. 


1 variable=\ 
2 echo "Svariable" 
3 # Will not work gives an error messag 
4 4 test.sh: command not found 
5 # A "naked" escape cannot safely be assigned to a variable. 
6 4 
7 4 What actually happens here is that the "\" escapes the newline and 
8 #+ the effect is variable-echo "Svariable" 
9 #+ invalid variable assignment 
10 
11 variable=\ 
12 23skidoo 
13 echo "Svariable" #  23skidoo 
14 # This works, since the second line 
ILS) #+ is a valid variable assignment. 


16 
17) 
18 
19 
20 
2 
22 
23 
24 
25 
26 
2 
28 
29 
30 
Sul 
32 
33 
34 
35) 


variable=\ 


d x^ escape followed by space 
echo "Svariable" # space 
variable=\\ 


echo "$variable" # \ 


variable=\\\ 

echo "Svariable" 

# Will not work gives an error messag 

# test.sh: \: command not found 

# 

# First escap scapes second one, but the third one is left "naked", 
#+ with same result as first instance, above. 


variable=\\\\ 

echo "Svariable" HAN 
# Second and fourth escapes escaped. 
wr JUgabsr sel Ogle. 


Escaping a space can prevent word splitting in a command's argument list. 


al 
il 


il 
2 
E 
4 
5 
6 
7 
8 


file 
# Li 


_list="/bin/cat /bin/gzip /bin/more /usr/bin/less /usr/bin/emacs-20.7" 
st of files as argument(s) to a command. 


s Nolo ice sabia So) tie lisit- cwe bist cubo 


des s 


1 /usr/X11R6/bin/xsetroot /sbin/dump $file list 


cho 


9 # What happens if we escape a couple of spaces? 
10 ls -1 /usr/X11R6/bin/xsetroot\ /sbin/dump\ $file list 


1 
2 


# Er 


# 


ror: the first three files concatenated into a single argument to 'ls -1' 
because the two escaped spaces prevent argument (word) splitting. 


The escape also provides a means of writing a multi-line command. Normally, each separate line constitutes a 
different command, but an escape at the end of a line escapes the newline character, and the command 
sequence continues on to the next line. 


pe 


£ 


al 


1 
2 
3 
4 
S 
6 
j 
8 
9 
0 


Cou 
(cd 
Re 
bu 


As 
tar 
bar 
Se 
(aE 


//isyoeuereev/olbxtexeE e due trar Cie = s » || \ 
/dest/directory && tar xpvf -) 

peating Alan Cox's directory tree copy command, 
t split into two lines for increased legibility. 


an alternative: 
er = -C /sommce/cimsctory s | 
xpvf C /dest/directory 
e note below. 
hanks, Stéphane Chazelas.) 


If a script line ends with al, a pipe character, then a V an escape, is not strictly necessary. It is, however, 
good programming practice to always escape the end of a line of code that continues to the following 
line. 


or 19) cp (xj Gal de (eS). [R9 [5 


' foo 
# No difference yet. 


#foo 
#bar 


echo 
echo foo\ 


bar # Newline escaped. 
#foobar 


echo 


Mop p opp ppppÀDGÀG:! 
O 0 0 100 50NHPDÓÀO 


echo "foo\ 
bar" # Same here, as \ still interpreted as escape within weak quotes. 
#foobar 


No ON 
[R3 [2 


NN WN 
(Ont gm. (G8) 


echo 


N N 
10 


eclme "fee 

bar' # Escape character \ taken literally because of strong quoting. 
#£00\ 

#bar 


N 
[99] 


WN 
C «o 


Si 
32 # Examples suggested by Stéphane Chazelas. 


io 
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Chapter 6. Exit and Exit Status 


... there are dark corners in the Bourne shell, and 
people use all of them. 


--Chet Ramey 
The exit command terminates a script, just as in a C program. It can also return a value, which is available to 
the script's parent process. 


Every command returns an exit status (sometimes referred to as a return status or exit code). A successful 
command returns a 0, while an unsuccessful one returns a non-zero value that usually can be interpreted as an 
error code. Well-behaved UNIX commands, programs, and utilities return a 0 exit code upon successful 
completion, though there are some exceptions. 


Likewise, functions within a script and the script itself return an exit status. The last command executed in the 
function or script determines the exit status. Within a script, an exit nnn command may be used to deliver 
an nnn exit status to the shell (nnn must be an integer in the 0 - 255 range). 


8^) When a script ends with an exit that has no parameter, the exit status of the script is the exit status of the 
last command executed in the script (previous to the exit). 


#!/bin/bash 


COMMAND 1 


COMMAND LAST 
9 # Will exit with status of last command. 


11 exit 
The equivalent of a bare exit is exit $? or even just omitting the exit. 


#!/bin/bash 


COMMAND 1 


COMMAND. LAST 


# Will exit with status of last command. 


B O Ne) foe) TOY Tea) CS (a) [soe EY 


eH 


exit $? 


#!/bin/bash 


COMMAND 1 


COMMAND_LAST 


Wey ceo) Sa] fen) (Gn) des (69) jest TS 


# Will exit with status of last command. 


$? reads the exit status of the last command executed. After a function returns, $? gives the exit status of the 
last command executed in the function. This is Bash's way of giving functions a "return value." [1] 


Following the execution of a pipe, a $? gives the exit status of the last command executed. 


After a script terminates, a $? from the command-line gives the exit status of the script, that is, the last 
command executed in the script, which is, by convention, 0 on success or an integer in the range 1 - 255 on 


error. 


Example 6-1. exit / exit status 


[s 
O Wey (er x rex Gal des GH [eS p 


|o 
NH 


ILS 
14 
LS 


#!/bin/bash 
echo hello 
echo $? # Exit status 0 returned because command executed successfully. 
lskdf # Unrecognized command. 
echo $? # Non-zero exit status returned because command failed to execute. 
echo 
exit ii 3} # Will return 113 to shell. 
# To verify this, type "echo $?" after script terminates. 
# By convention, an 'exit 0' indicates success, 
#+ while a non-zero exit value means an error or anomalous condition. 


$2 is especially useful for testing the result of a command in a script (see Example 16-35 and Example 16-20). 


$^) The !, the logical not qualifier, reverses the outcome of a test or command, and this affects its exit status. 


Example 6-2. Negating a condition using ! 


ererererrrrrr rrr 
XO 00 1 OY O1 4» (Q0 I) LPS O (o O0 -1 OY O1 4 w I2 HS 


NNN 
Ri E] 


23 
24 
25 
26 


true # The "true" builtin. 
echo "exit status of \"true\" = $2?" # 0 
! true 
echo) Veit status oF VM! true = ce! # 1 

Note that the "!" needs a space between it and the command. 

'true leads to a "command not found" error 

The '!' operator prefixing a command invokes the Bash history mechanism. 
true 
!true 


No error this time, but no negation either. 
It just repeats the previous command (true). 


# 
Preceding a _pipe_ with ! inverts the exit status returned. 
ls | bogus, command # bash: bogus command: command not found 
echo $? # 127 


! ls | bogus, command # bash: bogus command: command not found 
echo $? # 0 

# Note that the ! does not change the execution of the pipe. 

# Only the exit status changes. 

# # 


27 
28 # Thanks, Stéphane Chazelas and Kristopher Newsome. 


d Certain exit status codes have reserved meanings and should not be user-specified in a script. 


Notes 


[1] Inthose instances when there is no return terminating the function. 
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Chapter 7. Tests 


Every reasonably complete programming language can test for a condition, then act according to the result of 
the test. Bash has the test command, various bracket and parenthesis operators, and the if/then construct. 


7.1. Test Constructs 


* An if/then construct tests whether the exit status of a list of commands is 0 (since 0 means "success" 
by UNIX convention), and if so, executes one or more commands. 

* There exists a dedicated command called [ (left bracket special character). It is a synonym for test, 
and a builtin for efficiency reasons. This command considers its arguments as comparison expressions 
or file tests and returns an exit status corresponding to the result of the comparison (0 for true, 1 for 
false). 

e With version 2.02, Bash introduced the [[ ... ]] extended test command, which performs comparisons 
in a manner more familiar to programmers from other languages. Note that [[ is a keyword, not a 
command. 


Bash sees [[ $a -lt $b ]] asa single element, which returns an exit status. 

e. 
The (C... )) and let ... constructs also return an exit status, according to whether the arithmetic 
expressions they evaluate expand to a non-zero value. These arithmetic-expansion constructs may 
therefore be used to perform arithmetic comparisons. 


iL (4 0 ee X ») # Logical AND 
2 echo $? # 1 Boobs 
3 # And so 
4 let "num = (( 0 && 1 ))" 
5 echo $num # 0 
6 # But 
7 let "num = (( 0 && 1 ))" 
8 echo $? # 1 X 
9 
10 
qoas P ORC TETUR i) # Logical OR 
12 echo S? # 0 os 
13 4 
La Jüewe Vion = (1 2100) [n dd jy 
15 echo $num # 1 
i$ ec Vus = (C 20909 |i) di 3" 
17 echo $? # 0 APIS 
RS 
19 
20 (¢ 290 | La y) # Bitwise OR 
21 echo $? # 0 TURN 
QoS d ers 
AS dee “iain = (( 200 | id »jW" 
24 echo $num # 203 
Z5. ee Wsenbw = (( 20 |) bb pw 
26 echo $? # 0 Pea E 
27 


28 # The "let" construct returns the same exit status 
29 #+ as the double-parentheses arithmetic expansion. 


An if can test any command, not just conditions enclosed within brackets. 


if cmp a b &> /dev/null # Suppress output. 
then echo "Files a and b are identical." 
else echo "Files a and b differ." 

ial 


# The very useful "if-grep" construct: 
# 
Iie Grep c Bashi Tile 

then echo "File contains at least one occurrence of Bash." 


ERROR OCCURRI 


Jg acu 

DTE 

12 word-Linux 

13 letter sequence-inu 

14 if echo "Sword" | grep -q "S$letter sequence" 
15 # The "-q" option to grep suppresses output. 
16 then 

17 echo "$letter sequence found in $word" 

18 else 

19 echo "$letter sequence not found in $word" 
20) ie aL 

Z3 

22 

23 if COMMAND WHOSE EXIT STATUS IS 0 UNLESS 

24 then echo "Command succeeded." 

25 else echo "Command failed." 

29 Eu 


e These last two examples courtesy of Stéphane Chazelas. 


Example 7-1. What is truth? 


#!/bin/bash 


# Tip: 


# If you're unsure of how a certain condition would evaluate, 
je (ene SHE cie ua SIE SSE s 


echo 


Cleo) Waresiestiave; eO 
# zero 


nie [ @ J 
then 
echo 
else 
echo 
I3. 


echo 


MQ) ass qiu s 


# Or else 


(9) skis} reise v 


# 0 is true. 


Seno VESLE, UTENTI 
# one 


KEI OTSE 
then 
echo 
else 
echo 
SEIk 


echo 


echo "Testing 


ane ab a] 
then 

Sebo "d. aks 
else 

cchor Vail als 
if aL 
echo 


echo "Testing 
mie [p ] 
then 
echo 
else 


DIL ake: tee we SU 


UE CSErSt TEASE 


iz dk, 3er true. 


Beal We " 


# minus 


trug 


false." 
# -1 is 


E LU NU 
# NU 


one 


TE Ue. 


TN "m 
LL 


(empty condition) 


NIBUS Steuer 


44 
45 
46 
47 
48 
49 
50 
Sl 
52 
53) 
54 
55 
56 
5 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
qi 
72 
13 
74 
US 
76 
y 
78 
T9 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
gi 
92 
93) 
94 
EIS 
96 
23 
98 
99 
100 
101 
102 
LOS 
104 
105 
106 
107 
108 
109 


echo "NULL is false." 


aL # NULL is false. 
echo 
seine Ves zya 


se [D xem ] # string 
then 
Schon Uieteuewelewn XenExeIHOYS] Sir saucer 
else 
echo "Random string is false." 
1E 3b # Random string is true. 
echo 
acing Venen; WU Seu NUM 


zs [ Sra 1 w^ AUSSIE Git Seas Ser daubULILo est c. ous 


# it's only an uninitialized variable. 

then 

echo "Uninitialized variable is true." 
else 

echo "Uninitialized variable is false." 
fi # Uninitialized variable is false. 
echo 
eeno Viestin WU cies Wee NU 


a [ m Yaya ] # More pedantically correct. 
then 
echo "Uninitialized variable is true." 
else 
echo "Uninitialized variable is false." 
i3 # Uninitialized variable is false. 
echo 
xyz- # Initialized, but set to null value. 
exime “Wesicaios Wm Ws UU 


xe [| =m Wissen i] 
then 
echo "Null variable is true." 
else 
echo "Null variable is false." 
acak # Null variable is false. 
echo 
# When is "false" true? 


echo Wiresiesiov; ^y Eee uU 


av. qp. ied. q # It seems that "false" is just a string. 
then 
eho W\ Vireule~e\ ais iereule, U Gish shovel able TOSC CVG, 
else 
eyeloyo) WwUTEEdlexe NU ahs, Teese ^ 
icak # "false" is true. 
echo 
echo "Testing \"\S$false\"" # Again, uninitialized variable. 
ave dp SEES “1 
then 


110 echo UWXUNXSTalssW! is trua,” 

111 else 

112 Ero W\W\sireae\W ais deevlbeyess w 

i13 i 4 "Sfalse" is false. 

114 # Now, we get th xpected result. 
11,5) 

116 # What would happen if we tested the uninitialized variable "Strue"? 
dL y) 

118 echo 

i19 

120) exate 10) 


Exercise. Explain the behavior of Example 7-1, above. 


su || COmelriciom=ceUS || 
then 

command 1 

command 2 


else # Or else 
# Adds default code block executing if original condition tests false. 
command 3 
9 command 4 
10 
Wil jeu 


1 
2 
3 
4 
5 
6 
7 
8 


$^) When if and then are on same line in a condition test, a semicolon must terminate the if statement. Both if 
and then are keywords. Keywords (or commands) begin statements, and before a new statement on the 
same line begins, the old one must terminate. 


À if | =x "Sri lanana” ]g (mem 


Else if and elif 


elif 
elif is a contraction for else if. The effect is to nest an inner if/then construct within an outer one. 


ax || Ce@mellicisomil | 
then 
command1l 
command2 
command3 
elif [ condition2 ] 
# Same as else if 
then 
command4 
command5 
else 
L2 default-command 
US) ss 


fa 
(& We) (we x rex; Gal des (9 [ES d 


[2s 
[un 


Theif test condition-true constructis the exact equivalent of if [ condition-true ].As 
it happens, the left bracket, [ , is a token [1] which invokes the test command. The closing right bracket, ] , in 
an if/test should not therefore be strictly necessary, however newer versions of Bash require it. 


$^; The test command is a Bash builtin which tests file types and compares strings. Therefore, in a Bash 
script, test does not call the external /usr/bin/test binary, which is part of the sh-utils package. 
Likewise, [ does not call /usr/bin/ [, which is linked to /usr/bin/test. 


bash$ type test 
test is a shell builtin 
bash$ type '[' 
[ dus a Glad. Jost ite alin. 
bash$ type '[[' 
[[ is a shell keyword 
bash$ type ']]' 
]] is a shell keyword 
bash$ type ']' 
pashi Tey |] XoxE Found 


If, for some reason, you wish to use /usr/bin/test ina Bash script, then specify it by full 
pathname. 


Example 7-2. Equivalence of test, /usr/bin/test, [ ],and /usr/bin/ [ 


1 #!/bin/bash 

2 

3 echo 

4 

G. Adr eee cr. Velu 

6 then 

7 echo "No command-line arguments." 

8 else 

9 echo "First command-line argument is $1." 
IQ) 33i 
Li 
12 echo 
alli 
id aie /usr/oln/ test =z UST # Equivalent to "test" builtin. 
dS} ne te PS On ON A # Specifying full pathname. 
16 then 
JL) echo "No command-line arguments." 
18 else 
Lg) echo "First command-line argument is $1." 
2) Ea 
24 
22 echo 
23 
Zu caves posa URS n # Functionally identical to above code blocks. 
PASE OSEE ES RSS Bhougl work. Dub. 
26 #+ Bash responds to a missing close-bracket with an error message. 
27 then 
28 echo "No command-line arguments." 
29 else 
30 echo "First command-line argument is $1." 
Sil stat 
S2 
33 echo 
34 
35 
SE ase Jae AoA A USD. 3 # Again, functionally identical to above. 
S) x stie Puero || =e "Sq # Works, but gives an error messag 
38 # # Note: 
39 # This has been fixed in Bash, version 3.x. 
40 then 

41 echo "No command-line arguments." 

42 else 

43 echo "First command-line argument is $1." 

44 fi 

45 

46 echo 


47 
4B ex 0 


The [[ ]] construct is the more versatile Bash version of [ ]. This is the extended test command, adopted from 
ksh88. 


x k OK 


No filename expansion or word splitting takes place between [[ and ]], but there is parameter expansion and 
command substitution. 


file=/etc/passwd 


xi [f| e Site: JT 
then 
echo "Password file exists." 
fi 
Using the [[ ... ]] test construct, rather than [ ... ] can prevent many logic errors in scripts. For example, the 
&&, ll, <, and > operators work within a [[ ]] test, despite giving an error within a [ ] construct. 


(ox (Om ge (3 [9 [S 


Arithmetic evaluation of octal / hexadecimal constants takes place automatically within a [[ ... ]] construct. 


# [[ Octal and hexadecimal evaluation ]] 
# Thank you, Moritz Gronbach, for pointing this out. 


decimal=15 
octal=017 # = 15 (decimal) 
hex=0x0f # = 15 (decimal) 


1 
2 
3 
4 
5 
6 
7 
8 
9) zi | YScleeimal’ -sc "Seed || 
10 then 
i echo "Sdecimal equals Soctal" 
2 else 
3 echo "Sdecimal is not equal to $octal" sr LS if mot cepe to (9L 
ab aea # Doesn't evaluate within [ single brackets ]! 
5 
6 
Ji 
8 


s (|| WexeleguomedL" ee US Oxed 1] 
18 then 
Ag echo "$decimal equals $octal" # 15 equals 017 
20 else 
2l echo "$decimal is not equal to $octal" 
2 apu # Evaluates within [[ double brackets ]]! 
25 
24 ae [|| "SeleesunsdL" see, “Sines” pg] 
25 then 
26 echo "$decimal equals $hex" # 15 equals Ox0f 
27 else 
28 echo "Sdecimal is not equal to $hex" 
29) seat # [[ Shexadecimal ]] also evaluates! 


£^; Following an if, neither the test command nor the test brackets ( [ ] or [[ ]] ) are strictly necessary. 


al 
2 
3 
4 
5 


dir=/home/bozo 


sLt (ul Ugebusw 2o/claw/meilile eNe # "2>/dev/null" hides error messag 
echo "Now in $dir." 
else 


6 echo Cante change to Selir,” 


T EL 


The "if COMMAND" construct returns the exit status of COMMAND. 


Similarly, a condition within test brackets may stand alone without an if, when used in combination with 


a list construct. 


The (()) construct expands and evaluates an arithmetic expression. If the expression evaluates as zero, it 
returns an exit status of 1, or "false". A non-zero expression returns an exit status of 0, or "true". This is in 


1 vari1-20 
2 var2-22 


3 || "exei" me USE 


4 
5 home-/home/bozo 


6 [ -d "Shome" ] || 


] ce echo "$varl is not equal to Svara” 


echo "Shome directory does not exist." 


marked contrast to using the test and [ ] constructs previously discussed. 


Example 7-3. Arithmetic Tests using (( )) 


#!/bin/bash 
# arith-tests.sh 
# Arithmetic tests. 


T 
2 
B 
4 
5 # The (( ... )) construct evaluates and tests numerical expressions. 
G v HIE SEREWS Opposite Erow [ oos || GOomsuXU ! 
7 
9 (( @ )) 
3 ceho Visle Situs OIE QU PPV xg SP." il 
10 
WE (4 X ») 
12 ceho Vrait statas (E ab Sy We ESS e 0 
13 
TA ST Aa yay true 
15 exeo "pbi statues OË 5 S AMNS sg Sp. 0 
16 
Ly (( 9» 95) false 
18 ceho "imu SEEMS (UE 5 > 9 yw" as Se.” 1 
19 
20 (( 5 == Vy true 
i eeno "bibis SESAME (UE 5 == 5 )])\ r is S2." 0 
22 4$ (( 5 = 5 )) gives rror messag 
23 
Ba (¢ & = & }) 0) 
Z5. eeno "exilio SEAMS Or Gs, )) e SI 1 
26 
297 (( 9 7 4) Division Coko 
26 Seine) Ue Statue OF Bf db pw ses VES 0 
29 
S0 (( X f 2 yy) Division result « 1. 
3iL eho Ymic statues OF d 2 »ywNW aum $99,V Rounded off to Oa 
EZ T 
33 
S4 (( 1 7 © )) 225/els mm Illegal division by 0. 
35 EA E 
S6 echo Umi satus OE WU 4 / 9 NE xs Se. 1 
37 
38 What effect does the "2>/dev/null" have? 
38 What would happen if it were removed? 
40 Try removing it, then rerunning the script. 
Ai 
42 + 


43 

4d s (( sos JJ) also tsen im ain Decline ESS. 
45 

46 varl=5 

47 var2=4 

48 

49) aw ( varl > wase2 j) 

50 then #^ i Note: Not $varl, $var2. Why? 
Sil echo "Svarl is greater than $var2" 

52 EI # 5 is greater than 4 

53 

54 exit 0 


Notes 


[1] A token is a symbol or short string with a special meaning attached to it (a meta-meaning). In Bash, 
certain tokens, such as [ and . (dot-command), may expand to keywords and commands. 
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7.2. File test operators 


Returns true if... 


-e 
file exists 
-à 
file exists 
This is identical in effect to -e. It has been "deprecated," [1] and its use is discouraged. 
-f 
file is a regular file (not a directory or device file) 
-S 
file is not zero size 
-d 
file is a directory 
-b 
file is a block device 
-C 
file is a character device 
1 device0="/dev/sda2" # / (root directory) 
2 ae | Sy YSclewieceO™ | 
3 then 
4 echo "Sdevice0 is a block device." 
5 Ed 
6 
7 s J/dew/ssca2 ise a block cewilee. 
8 
B 
10 
11 devicel="/dev/ttysi" # PCMCIA modem card. 
12 if [| -c "Sdevicel" ] 
13 then 
14 echo "Sdevicel is a character device." 
15 £i 
16 
17 4 /dev/ttyS1 is a character device. 
-P 
file is a pipe 
-h 
file is a symbolic link 
-L 
file is a symbolic link 
-S 
file is a socket 
-t 
file (descriptor) is associated with a terminal device 
This test option may be used to check whether the stdin [ -t 0 ]orstdout [ -t 1 ]ina 
given script is a terminal. 
-r 
file has read permission (for the user running the test) 
-W 


file has write permission (for the user running the test) 


file has execute permission (for the user running the test) 


-8 
set-group-id (sgid) flag set on file or directory 
If a directory has the sgid flag set, then a file created within that directory belongs to the group that 
owns the directory, not necessarily to the group of the user who created the file. This may be useful 
for a directory shared by a workgroup. 

-u 
set-user-id (suid) flag set on file 
A binary owned by root with set-user-idflag set runs with root privileges, even when an 
ordinary user invokes it. [2] This is useful for executables (such as pppd and cdrecord) that need to 
access system hardware. Lacking the suid flag, these binaries could not be invoked by a non-root 
user. 
iesus TE L root 176236 Oce 2 2000 Jüg/stsm/prel 
A file with the suid flag set shows an s in its permissions. 

-k 
sticky bit set 
Commonly known as the sticky bit, the save-text-mode flag is a special type of file permission. If a 
file has this flag set, that file will be kept in cache memory, for quicker access. [3] If set on a 
directory, it restricts write permission. Setting the sticky bit adds a t to the permissions on the file or 
directory listing. 
drwxrwxrwt T TOO 1024 May 19 21:26 tmp/ 
If a user does not own a directory that has the sticky bit set, but has write permission in that directory, 
she can only delete those files that she owns in it. This keeps users from inadvertently overwriting or 
deleting each other's files in a publicly accessible directory, such as / tmp. (The owner of the 
directory or root can, of course, delete or rename files there.) 

-O 
you are owner of file 

-G 
group-id of file same as yours 

-N 
file modified since it was last read 

fl -nt f2 
file f1 is newer than f2 

fl -ot f2 
file f1 is older than £2 

fl -ef f2 


files £1 and £2 are hard links to the same file 


"not" -- reverses the sense of the tests above (returns true if condition absent). 


Example 7-4. Testing for broken links 


1 #!/bin/bash 
2 4 broken-link.sh 
3 4 Written by L bigelow <ligelowbee@yahoo.com> 


4 Used in ABS Guide with permission. 
5 
6 A pure shell script to find dead symlinks and output them quoted 
7 #+ so they can be fed to xargs and dealt with :) 
8 #+ eg. sh broken-link.sh /somedir /someotherdir|xargs rm 
9 
10 This, however, is a better method: 
LL 
12 find "somedir" -type 1 -printO0|wN 
4.3 xargs -r0 file|\ 
14 grep "broken symbolic"| 
15 sed -e 's/*\|: *broken symbolic.*$/"/g' 
16 
17 #+ but that wouldn't be pure Bash, now would it. 
18 Caution: beware the /proc file system and any circular links! 
19 HEHEHE HE HE EEE EE HH HE HE TE FE FE EE HE HE EE RE EE E TE FE EE EE ER EE E E E EE E E E RE EE 
20 
Bil 
2 If no args are passed to the script set directories-to-search 
23 #+ to current directory. Otherwise set the directories-to-search 
24 #+ to the args passed. 
25 HEHEHE EEE RE EERE HE EE 
26 
27 $4 -eq 0 ] && directorys= pwd || directorys=S@ 
28 
29 
30 Setup the function linkchk to check the directory it is passed 
31 #+ for files that are links and don't exist, then print them quoted. 
32 If one of th lements in the directory is a subdirectory then 
33 #+ send that subdirectory to the linkcheck function. 
34 TER ERE EE 
35 
3 imke () 4 
37 tow Glemene in Si/ee clo 
38 [ -h "Selement" -a ! "Selement" ] && echo \"Selement\" 
39 [ -d "Selement" ] && linkchk Selement 
40 i? (Qi courses, hat tesca io: symloolie lumi, "gl" tor ciesctomy. 
41 done 
42 ) 
43 
44 # Send each arg that was passed to the script to the linkchk() function 
Wy Gear abit SHE 30S a valid Glatecioy. TE wit, inem goes iem rror messag 
46 #+ and usage info. 
47 FHEEEEERE EE EEE EE EE 
48 for directory in $directorys; do 
49 1e [ ed Sclimecicoiay ] 
50 then linkchk $directory 
Su else 
52 echo "Sdirectory is not a directory" 
53 echo YWsacges SO chirl Cira sgsY 
54 fti 
55 done 
56 
57 ezit SP 


Example 30-1, Example 11-7, Example 11-3, Example 30-3, and Example A-1 also illustrate uses of the file 
test operators. 


Notes 


[1] Perthe 1913 edition of Webster's Dictionary: 


1 Deprecate 


To pray against, as an evil; 
to seek to avert by prayer; 
to desire the removal of; 
to seek deliverance from; 
to express deep regret for; 
to disapprove of strongly. 


to co —] oo) Ol d WS 


[2] Be aware that suid binaries may open security holes. The suid flag has no effect on shell scripts. 


[3] On Linux systems, the sticky bit is no longer used for files, only on directories. 
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7.3. Other Comparison Operators 


A binary comparison operator compares two variables or quantities. Note that integer and string comparison 
use a different set of operators. 


integer comparison 


-eq 
is equal to 
if [ " $a" —eq " $b" ] 
-ne 
is not equal to 
if [ " $a" —ne " $b" J 
-gt 
is greater than 
if [ " $a" -gt " $b" ] 
-ge 
is greater than or equal to 
if [ " $a" -ge " $b" ] 
-It 
is less than 
if [ " $a" -lt " $b" ] 
-le 
is less than or equal to 
if [ " $a" -le " $b" ] 
« 
is less than (within double parentheses) 
(("Sa" « "$b")) 
<= 
is less than or equal to (within double parentheses) 
(("Sa" <= "Sb") ) 
> 
is greater than (within double parentheses) 
(("Sa" > "Sb") ) 
>= 


is greater than or equal to (within double parentheses) 
(("Sa" >= "Sb") ) 


string comparison 


is equal to 


if [ " $a" = i" $b" J 
is equal to 
if [ "Sa" == "$b" ] 


This is a synonym for =. 


$^) The == comparison operator behaves differently within a double-brackets test than within 


single brackets. 


i [i Sa ee 2 ]l # True if $a starts with an "z" (pattern matching). 
2. [L[ Sa == "ge qup se Wes ai Sa as Sewell to mw (ilsiveicad meceni ine) s 

3 

4 [ Sa == z* ] # File globbing and word splitting take place. 

5 | Wav == Ware j s ree i Sa as ctal to m (ilnieeieall meaieclamning)) . 

6 

7 # Thanks, Stéphane Chazelas 


is not equal to 

if [ "Sa" != "$b" ] 

This operator uses pattern matching within a [[ ... ]] construct. 
is less than, in ASCII alphabetical order 

if [[ "$a" « "$b" ]] 

if [ "$a" \< "$b" ] 

Note that the "«" needs to be escaped withina [ ] construct. 
is greater than, in ASCII alphabetical order 

if [[ "$a" > "$b" ]] 

if [ "$a" \> "$b" ] 

Note that the ">" needs to be escaped within a [ ] construct. 
See Example 27-11 for an application of this comparison operator. 
string is null, that is, has zero length 


String='' # Zero-length ("null") string variable. 


af | =z VSStzaiue" | 
then 

een UXSSESE ag moli, 7 
else 

cine A Strine She NOL mul, 
iab 7 Geri ws mil. 


(sey =| fey) (rl Ss (ex) i [3 


string is not null. 


The -n test requires that the string be quoted within the test brackets. Using an 
unquoted string with / -z, or even just the unquoted string alone within test brackets 
(see Example 7-6) normally works, however, this is an unsafe practice. Always quote a 
tested string. [1] 


Example 7-5. Arithmetic and string comparisons 


#!/bin/bash 


# Here "a" and "b" can be treated either as integers or strings. 
# There is some blurring between the arithmetic and string comparisons, 
#+ Since Bash variables are not strongly typed. 


# Bash permits integer operations and comparisons on variables 
#+ whose value consists of all-integer characters. 
# Caution advised, however. 


echo 


a [ "Sa" sme "S5" ] 
then 
exco "Sa is moi eewall io $o” 
echo "(arithmetic comparison)" 
ipa 


NPRPRPRPRP PPP PY 
O (o 0» -1 O Ui i (). M HP. O xo 0 -1 O UI 4 Q Io E 


N N 
N He 


echo 


N N 
B W 


ae [p USQw Je USQW T 
then 
exo "Sa sg moe egal ito S," 
echo "(string comparison)" 
# TATUM mST 
JURE AS Clea 52 S ASCIT O3 
iral 


NNN 
~= fen) (Oni 


N 
[99] 


# In this particular instance, both "-ne" and "!-" work. 


echo 


C9 CO CO CO CO CO CO DN 
Gy GI fF ON EF eS © 


exit 0 


Example 7-6. Testing whether a string is null 


1 #!/bin/bash 

2 str-test.sh: Testing null strings and unquoted strings, 

3 #+ but not strings and sealing wax, not to mention cabbages and kings 
4 

5 Using BEES ER] 

6 

7 If a string has not been initialized, it has no defined value. 

8 This state is called "null" (not the same as zero!). 

9 
19 ae [ -=m Smeg | # stringl has not been declared or initialized. 
11 then 


1.2 eeno Weering Weg WU as moe imul. 
13 else 


14 


eeno VSrrine NYST ringi” als mall, Y 
Ral, # Wrong result. 
# Shows $stringl as not null, although it was not initialized. 


echo 


# Let's try it again. 


aie [| m Yesera J| xx Wane cima, Secrest, L5 Croce, 
then 
echo Wishescakioves Werring is mor mali w 
else 
CENO VSTAL VASEN LS L A 
FIL # Quote strings within test brackets! 
echo 
aie [| Serene | # This time, S$stringl stands naked. 
then 
echo Wishewaliove; erring” is not eNDLILIL sU 
else 
Scag Serine WU esEsbe NU her iil. 7 
1E al # This works fine. 
# The [ ... ] test operator alone detects whether the string is null. 
# However it is good practice to quote it (i£ [| "Sstringl" ||). 


# 
# As Stephane Chazelas points out, 


d aie [| Genseabagdh | has one argument, "]" 
# aise [| Veeringe A | Ines cwo euceibineimes, TAE (wow VSSE il wel WW 
echo 


stringl=initialized 


sit [p Sexe T # Again, S$stringl stands unquoted. 
then 
exclave, Wisheaciiave; Werring is nor mali, w 
else 
eeno “Givienine NYSS ae) maL 
imal, # Again, gives correct result. 
i? Stemi, slic te berte to (obo rie abc (YU Smicieiine il), locate 


stringl="a = b" 


se | SSieiedinesl | # Again, Sstringl stands unquoted. 
then 
echo Werring \Wisiziestino al) als) nor mail, Y 
else 
echo WSivwealiye; \WYsicieniovel NU ie L 
iti # Not quoting "Sstringl" now gives wrong result! 
exit 0 # Thank you, also, Florian Wisser, for the "heads-up". 


Example 7-7. zmore 


de X» [5 [2 


#!/bin/bash 
# zmore 


# View gzipped files with 'more' filter. 


E NOARGS-65 
E NOTFOUND-66 
E NOTGZIP-67 


ie | Sie ea © 1 w sane Guect ass cds [ m WSL TI 
SL (ein Gralsic, but be empty: i zmore UU fuse actos) 
then 

echo "Usage: `basename $0! filename" >&2 

# Error message to stderr. 

exit $E NOARGS 

# Returns 65 as exit status of script (error code). 
ie aL 


I 


PRPPRPP PY 
O0 -1 OY Ui 4 CQ). I9. IB. O (0 00 -1 Oy O1 


19 filename-$1 


20 

All ait || | =i Wiitsilemenne” | # Quoting $filename allows for possible spaces. 
22 then 

29 echo "File $filename not found!" >&2 # Error message to stderr. 

24 exit $E NOTFOUND 

Z5 i3 

26 

27 if [ S${filename##*.} != "gz" J 

28 # Using bracket in variable substitution. 

29 then 


30 Gene "ssl: Sil is moe ci Cmijsjxecl iene!” 
Sil exit SE_NOTGZIP 
32 seal 


3 soe Sil || mons 


36 4 Uses the 'more' filter. 
37 4 May substitute 'less' if desired. 


99 cou $7 # Script returns exit status of pipe. 
40 # Actually "exit $?" is unnecessary, as the script will, in any case, 
41 #+ return the exit status of the last command executed. 


compound comparison 


-à 
logical and 


expl -a exp2 returns true if both expl and exp2 are true. 
logical or 
expl -o exp2 returns true if either expl or exp2 is true. 
These are similar to the Bash comparison operators && and Il, used within double brackets. 


iL [| comchiciomil e comelicicma2 || | 


The -o and -a operators work with the test command or occur within single test brackets. 


i ase [ YSexgoeil” =a WSexgoree” | 

2 then 

S echo "Both exprl and expr2 are true." 
4 else 

E echo "Either exprl or expr2 is false." 
G EL 


d But, as rihad points out: 


i i eee i | se [| =a Y echo tius 1-52 V" | # true 

2 L eec 2 ]| && | =a " esmao ie Ira V ] # (no output) 

3 ^^^^^^^ False condition. So far, everything as expected. 

4 

5 However 

6 JL ep 2 -a em echo tue Ios? V j # true 

7 ARARAS Malee Comechkelon, Se, wa "ixi" quiu 

8 

E Is it because both condition clauses within brackets evaluate? 
iLO) | d -ee 2 && =m VU Seine cite leca V Y # (no output) 
tal NO, devenus "I TOE Eo 
12 
13 Apparently && and || "short-circuit" while -a and -o do not. 


Refer to Example 8-3, Example 27-17, and Example A-29 to see compound comparison operators in action. 
Notes 


[1] As S.C. points out, in a compound test, even quoting the string variable might not suffice. [ -n 


"$string" -o "Sa" = "$b" ] may cause an error with some versions of Bash if $string is 
empty. The safe way is to append an extra character to possibly empty variables, [ "x$string" !- 
x -o "xSa" = "xSb" ] (the "x's" cancel out). 
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7.4. Nested if/then Condition Tests 


Condition tests using the i £/then construct may be nested. The net result is equivalent to using the && 
compound comparison operator. 


1 a=3 

2 

3 aie [| “Sal gu © T 

4 then 

9 mie | Sav site 5 

6 then 

7 echo "The value of \"a\" lies somewhere between 0 and 5." 
8 dE 3L 

9 Ea 
10 
11 # Same result as: 
12 
la ase [ "Saw e 0 | ee [| "Sa" ke 5 
14 then 
15 echo "The value of \"a\" lies somewhere between 0 and 5." 
1G 3a 


Example 36-4 demonstrates a nested i £/then condition test. 
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7.5. Testing Your Knowledge of Tests 


The systemwide xinitrc file can be used to launch the X server. This file contains quite a number of if/then 
tests. The following is excerpted from an "ancient" version of xinitrc (Red Hat 7.1, or thereabouts). 


1 af [ —-f SHOME/.Xcllients ]; then 

2 exec $HOME/.Xclients 

3 eladft [| —f /etc/XTl/xrndt/Xclrents ]; then 

4 exec /etc/X11/xinit/Xclients 

5 else 

6 4 failsafe settings. Although we should never get here 

7 # (we provide fallbacks in Xclients as well) it can't hurt. 
8 xclock -geometry 100x100-5+5 & 

9 xterm -geometry 80x50-504150 & 
LO if [ -f /usr/bin/netscape -a -f /usr/share/doc/HTML/index.html ]; then 
iil netscape /usr/share/doc/HTML/index.html & 
UZ Fr 


19 fx 
Explain the test constructs in the above snippet, then examine an updated version of the file, 
/etc/X11/xinit/xinitrc, and analyze the if/then test constructs there. You may need to refer ahead to 
the discussions of grep, sed, and regular expressions. 
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Chapter 8. Operations and Related Topics 


8.1. Operators 


assignment 


variable assignment 


Initializing or changing the value of a variable 
All-purpose assignment operator, which works for both arithmetic and string assignments. 


1 var-27 
2 category-minerals # No spaces allowed after the "=". 


D Do not confuse the "=" assignment operator with the = test operator. 


1 # = as a test operator 

2 

3 sir | Hegre = Seriene T 

4 then 

5 command 

© i 

7 

Gg s aie [| MoxSgitriimeg il” = YWoxScieriing2 ] as sarei, 

9 #+ to prevent an error message should one of the variables be empty. 
10 # (The prepended "X" characters cancel out.) 


arithmetic operators 


+ 


"ek 


% 


plus 

minus 

multiplication 

division 

exponentiation 
1 # Bash, version 2.02, introduced the "**" exponentiation operator. 
: lert ra -on 3i u^ 3 UU 5 SB 
4 echo "z = $z" i) 4 = ALAS 


modulo, or mod (returns the remainder of an integer division operation) 


bash$ expr 5 % 3 
2 


5/3 = 1, with remainder 2 


This operator finds use in, among other things, generating numbers within a specific range (see 


Example 9-11 and Example 9-15) and formatting program output (see Example 27-16 and Example 
A-6). It can even be used to generate prime numbers, (see Example A-15). Modulo turns up 


surprisingly often in numerical recipes. 


Example 8-1. Greatest common divisor 


Mop pop opp PPP PY 
O (o 0» -1 O Ui i (). M PB. O xo 0 -1 O O1 4 Q Io E 


(69). GIES ss ESSE dS TS Shoo) 
[= ~) (ex Ce) Sal (ex (On des (er E» d 


w 
N 


P C0 C) CO WW CO Ww 
(c3 (We) (ge) c (ex Gi des © 


od 


C1 Or; Sb ga uam us 
[= ~S) Ne) (ee) | vey (On. dex. C» IS) d 


(Gmbh Xon rj. Conk Cal toni Gri (nl 
«o0 00-10) oO FWN 


Ov OY OY 0 
CTS) [AS 


!/bin/bash 
gcd.sh: greatest common divisor 


Uses Euclid's algorithm 


The "greatest common divisor" (gcd) of two integers 
+ is the largest integer that will divide both, leaving no remainder. 


Euclid's algorithm uses successive division. 


In each pass, 


(Glabwrabelewgao] «=== chyson 
Gliwigsoic <=== remeni volar 
until remainder = 0. 


The gcd = dividend, on the final pass. 


For an excellent discussion of Euclid's algorithm, see 
* Jim Loy's site, http://www.jimloy.com/number/euclids.htm. 


Argument check 
ARGS- 
E BADARGS-85 


2 


ade [| S a VSUNSCGSU | 
then 
echo "Usage: ~basename $0! first-number second-number" 
exit SE BADARGS 
ia 
# 
gcd () 
{ 
dividend-$1 # Arbitrary assignment. 
divisor=$2 #! It doesn't matter which of the two is larger. 
# Why not? 
remainder=1 # If an uninitialized variable is used inside 
#+ test brackets, an error message results. 
until [ "Sremainder" -eq 0 ] 
do up AASAAAETYS WIE) ISS “JOICENALOUISIY? TNU aZe 
let "remainder = $dividend % $divisor" 
dividend=Sdivisor # Now repeat with 2 smallest numbers. 
divisor-$remainder 
done # Euclid's algorithm 
} # Last Sdividend is the gcd. 
gee Sil S2 
behol exeo MEG) Cie Sil mue SA = Scliyarclamels exl 
# Exercises 
# (Sate, 
# 1) Check command-line arguments to make sure they are integers, 
#+ and exit the script with an appropriat rror message if not. 
# 2) Rewrite the gcd () function to use local variables. 
exit 0 


plus-equal (increment variable by a constant) 

let "var += 5" results in var being incremented by 5. 
minus-equal (decrement variable by a constant) 

times-equal (multiply variable by a constant) 

let "var *= 4" results in var being multiplied by 4. 
slash-equal (divide variable by a constant) 

mod-equal (remainder of dividing variable by a constant) 


Arithmetic operators often occur in an expr or let expression. 


Example 8-2. Using Arithmetic Operations 


#!/bin/bash 
# Counting to 11 in 10 different ways. 
n=l; echo =m “Sin Y 


eeng =m “Sin 4 


$m = Sm x 1)) 


cla m Wem W 


(4C om mam ar db y) 
# A simpler alternative to the method above. 


echo =m Vm Y 


NPRPRPP PPP PP 
O (o 0» -1 O Oi i (Q). M IB. O «(o 0 -1 O O1 i QI E 


n-$(($n * 1)) 


Zi echo -m Umum 4 

2 

2S} &[ m= Sm + X | 

24 # ":" necessary because otherwise Bash attempts 
ZG) qvr CO sbenrrescpoueeie YE in = Si r db || as A Clommuetavel. 
26 # Works even if "n" was initialized as a string. 
al eio =m Sia Y 


N 
[99] 


n-$[ $n + 1 ] 


# Thanks, Stephane Chazelas. 
echo =m Wisin 4 


# Now for C-style increment operators. 
# Thanks, Frank Wang, for pointing this out. 


let "mee" # let "++n" also works. 
ehg =m “Sin Y 


WWWWW CO CO CO CO CO Dd 
co! Ji OY CT SO GN ES oe © 


od 
[e] 


ete Uu e Sin 4e dL 5p dere Win a x dU — musti. wor ksi 


41 ":" necessary because otherwise Bash attempts 
#+ to interpret "$((n = $n + 1))" as a command. 


# Thanks, David Lombard, for pointing this out. 


# Works even if "n" was initialized as a string. 
#* Avoid this type of construct, since it is obsolete and nonportable. 


aL (ume )) # (( ++n )) also works. 
42 elo =m Wem WV 

43 

44 : $(( nt+ )) # : $(( ++n )) also works. 
AS eel =a Vem Us 

46 

A7 3 $[ mew | a B Sil sm ] also works 
48 eeing =m Vem Y 

49 

50 echo 

Bil 

52 exit 0 


$^; Integer variables in older versions of Bash were signed long (32-bit) integers, in the range of 
-2147483648 to 2147483647. An operation that took a variable outside these limits gave an erroneous 


result. 

1 echo S$BASH VERSION ils 314 

2 

3 a=2147483646 

4 echo "a = $a" a = 2147483646 

5 lee Wag-dqn Increment "a" 

6 echo "a = Sa" a = 2147483647 

7 let "at=1" increment "a" again, past the limit. 

8 echo "a = Sa" a = -2147483648 

9 ERROR: out of range, 
10 + and the leftmost bit, the sign bit, 
ial sh has been set, making the result negative. 


As of version >= 2.05b, Bash supports 64-bit integers. 


Bash does not understand floating point arithmetic. It treats numbers containing a decimal point as 
strings. 


a-1.5 


1 
2 
3 lewr Wa = Sa ap das ss IMOS, 

4 4$ t2.sh: let: b = 1.5 + 1.3: syntax error in expression 
5 

6 

3 


# (error tokem is "5 4e 1,3") 


echo "Ul = Slo" # b=1 
Use bc in scripts that that need floating point calculations or math library functions. 


bitwise operators. The bitwise operators seldom make an appearance in shell scripts. Their chief use seems to 
be manipulating and testing values read from ports or sockets. "Bit flipping" is more relevant to compiled 
languages, such as C and C++, which provide direct access to system hardware. 


bitwise operators 


<< 

bitwise left shift (multiplies by 2 for each shift position) 
<<= 

left-shift-equal 

let "var <<= 2" results in var left-shifted 2 bits (multiplied by 4) 
>> 


bitwise right shift (divides by 2 for each shift position) 
>>= 


right-shift-equal (inverse of <<=) 


& 
bitwise AND 
&- 
bitwise AND-equal 
| 
bitwise OR 
|= 
bitwise OR-equal 
bitwise NOT 
A 
bitwise XOR 
A= 


bitwise XOR-equal 


logical (boolean) operators 


NOT 
1 af [ ! —f SEILENAME | 
2 then 
3 

&& 

AND 
i, ac || Sexwweba3xuermul I ee [ Seomelslenem2 i 
2 i Seme ase as | Seomchicaemil =a Seomciucien2 ] 
S) s^  IeWEDUeN IS cere Git oora Comciicnemil ewe (exei Imoilel ewes. 
4 
5 at II sconchetioml dem. Seouektcacm2 Ii # Also works. 
6 # Note that && operator not permitted inside brackets 
Jg sou OE [ coo || CONSE OTa 


H) && may also be used, depending on context, in an and list to concatenate commands. 


OR 


ii | Seomchiciemil | |] | See@mciicica2 | 
# Same as: if [ $conditionl -o $condition2 ] 
i? Recim crue ii eltast Comeliicaeimill oie Cloimeliicsoim2 Moles Tues o o c 


wi II Scomcaicilonl || Scomcuticm2 || | # Also works. 
# Note that || operator not permitted inside brackets 
Wu Qu @ osse ] GGG. 


zu cxy (Ga) dE (59 soy [3 


$^; Bash tests the exit status of each statement linked with a logical operator. 


Example 8-3. Compound Condition Tests Using && and ll 


#!/bin/bash 


a=24 
b=47 


ey) (Gn dex (93. [S5 [55 


if [ "Sa" -eq 24 ] && [ "Sb" -eq 47 ] 


then 

echo "Test #1 succeeds." 
else 

echo "Test #1 fails." 
I3 


aF attempts to execute ' [ "$a" -eq 24 ' 
ar anc sells to ialieelilias; matenigo "] V. 


7 
8 
9 
0 
1 
1:2 
iis ERROR: ake [ “Sal eee Z4. pe Wou exer dy || 
4 
5) 
6 
7j Note: if [[ $a -eq 24 && $b -eq 24 ]] works. 
8 


T The double-bracket if-test is more flexible 
19 #+ than the single-bracket version. 


20 (The "&&" has a different meaning in line 17 than in line 6.) 
21 Thanks, Stephane Chazelas, for pointing this out. 
22 

23 

24 sait [ VEG" ew SE y [i (f "Se" ee 47 | 

25 then 

26 echo "Test 42 succeeds." 

27 else 

28 echo "Test #2 fails." 

29 t3 

30 

3d 

32 # The -a and -o options provide 


33 #+ an alternative compound condition test. 
34 # Thanks to Patrick Callahan for pointing this out. 


Sig! aise USaW.-ee 24 =e WS" see 47 | 
38 then 

39 echo "Test 43 succeeds." 

40 else 

41 echo "Test #3 fails." 

qo E 


45 if [| "$a" -eq 98 -o "Sb" -eq 47 | 
46 then 

47 echo "Test #4 succeeds." 

48 else 

49 echo "Test #4 fails." 

50 EL 


53 a-rhino 

54 b=crocodile 

Gi. ase [ ASe sun ] && [ "SO" S creocoouke ] 
56 then 

57 echo "Test #5 succeeds." 

58 else 

59 echo "Test #5 fails." 

GO ma 


62 e». (0) 


The && and ll operators also find use in an arithmetic context. 


bash$ echo $(( 1 && 2 )) $((3 && 0)) $((4 II 0)) $((0 II 0) 
i 9 i © 


miscellaneous operators 


Comma operator 


The comma operator chains together two or more arithmetic operations. All the operations are 
evaluated (with possible side effects. [1] 


dL lee “Wich = (45. x 35 7 c i, ds = 25» 

2, eine Uil = Sted EA ES EE TN 

3 # Here tl is set to the result of the last operation. Why? 
4 

5 


lige Wer = (fa = ©, 15 / 3))" 5 Sec Uma" e callewllace Ye2’ 
6 echo "t2 = BEZ a = Sa" # t2 = 5 a= 9 


The comma operator finds use mainly in for loops. See Example 11-12. 


Notes 


[1] Side effects are, of course, unintended -- and usually undesirable -- consequences. 


— 
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8.2. Numerical Constants 


A shell script interprets a number as decimal (base 10), unless that number has a special prefix or notation. A 
number preceded by a 0 is octal (base 8). A number preceded by 0x is hexadecimal (base 16). A 
number with an embedded # evaluates as BASE#NUMBER (with range and notational restrictions). 


Example 8-4. Representation of numerical constants 


!/bin/bash 
numbers.sh: Representation of numbers in different bases. 


Decimal: the default 

iet udeo == sy 

echo "decimal number = Sdec" # 32 
Nothing out of the ordinary here. 


Octal: numbers preceded by '0' (zero) 
lec Voce = (32V 
echo "octal number = Soct" # 26 


Expresses result in decimal. 


Hexadecimal: numbers preceded by 'Ox' or 'OX' 
let "hex = 0x32" 


NPRPRPPRP PPP PY 
O (o 00 -1 O Oi i (). NN IB. O xo 0 -1 O O1 iS CQ I0 


echo "hexadecimal number = Shex" # 50 
21 echo $((0x9abc)) it 39012 
22. s ix S double-parentheses arithmetic expansion/evaluation 
23 # Expresses result in decimal. 
24 
25 
26 
27 # Other bases: BASE#NUMBER 


N 
[99] 


# BASE between 2 and 64. 
# NUMBER must use symbols within the BASE range, see below. 


let "bin - 24111100111001101" 
echo "binary number = $bin" # 31181 


let "b32 = 32477" 


CO CO CO CO CO CO CO CO CO CO Dd 
We) Ge) Sal deny (Ont SS Gy) (SS) [E (e Keo} 


echo "base-32 number = $b32" # 231 
let "b64 = 64#@_" 
echo "base-64 number = $b64" # 4031 
40 # This notation only works for a limited range (2 - 64) of ASCII characters. 
41 4$ 10 digits + 26 lowercase characters + 26 uppercase characters + @ + — 
42 
43 
44 echo 
45 
46 echo $((36#zz)) $((2#10101010)) $((16#AF16)) $((5341aA)) 
47 # 1295 170 44822 3375 
48 
49 
50 Important note: 


Using a digit out of range of the specified base notation 
+ gives an error messag 


# 
# 
# 
# 


aon ul 
ass. (eX [9 qs 


55 let "bad oct = 081" 
56 # (Partial) error message output: 


57 4$ bad oct = 081: value too great for base (error token is "081") 

58 # Octal numbers use only digits in the range 0 - 7. 

5 

60 exit $? # Thanks, Rich Bartell and Stephane Chazelas, for clarification. 
61 


62 $ sh numbers.sh 
63 S cele SF 


64 $ 1 
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8.3. The Double-Parentheses Construct 


Similar to the let command, the (( ... )) construct permits arithmetic expansion and evaluation. In its simplest 
form, a-$(( 5 + 3 )) wouldsetato5 + 3,or 8. However, this double-parentheses construct is also a 
mechanism for allowing C-style manipulation of variables in Bash, for example, (( var++ )). 


Example 8-5. C-style manipulation of variables 


1 #!/bin/bash 
Z qu (gewsuss sh 
3 us AMusagexlLeNE3he] 4 weussbelole, (C-stEwie, wsime ies (( oc. JJ) GONmSEIEUKIE. 
4 
5 
6 echo 
7 
A a =] 2s iy) Setting a value, C-style, 
9 + with spaces on both sides of the "=". 
10 acing Ya (imitial valus) = Sal! s 23 
SIT 
T2 (t (aras Post-increment 'a', C-style. 
13 eleg. Ya lattar edt) = Sa” # 24 
14 
Lt a== Ji Post-decrement 'a', C-style. 
16 elo Wa (mures a  ) = Sal s 23 
aly) 
18 
19 (( ++a )) # Pre-increment 'a', C-style. 
20 echo Wa (Gees dej) = Sal # 24 
£a 
22 (( --a )) #  Pre-decrement 'a', C-style. 
23 eho Ya (mute a) = Se" ws 23 
24 
25 echo 
26 
27 AEE HE E HE E HE HE HE E HE E E E EEE EAE EEE EEE EEE EEE HE HE E HE HHH HE HHH HEHE HE HH 


N 
[99] 


# Note that, as in C, pre- and post-decrement operators 
#+ have different sid IEICE. 


echo "False" False 
echo "False" True 


n && echo "True" 
n && echo "True" 


n-1; let 
n-1; let 


# Thanks, Jeroen Domburg. 
HERE EEEE EEHHE HE HE HH EE HE HE EH HH HH HE EE HHH HH FE HE EE HEE 


WWWWWW CO CO CO CO Dd 
(ey (er c] xy Gal dex yl [S3 [5 €» Wo 


echo 
(( ie e cesare TREE y # C-style trinary operator. 
40 # WV SOSH 
Aq cohcomt tec abr then t Su eke t= INE QE 23) 
42 echo "t = $t " d t7 
43 
44 echo 
45 
46 
47 d 
48 # Easter Egg alert! 
49 d 
50 # Chet Ramey seems to have snuck a bunch of undocumented C-style 
51 #+ constructs into Bash (actually adapted from ksh, pretty much). 


52 # In the Bash docs, Ramey calls (( ... )) shell arithmetic, 

53 #+ but it goes far beyond that. 

54 # Sorry, Chet, the secret is out. 

55 

56 Ses elso "ies" amcl Vwim:le" locas using rise (( sa. )) Conget ttet. 
59 

58 4 These work only with version 2.04 or later of Bash. 

59 

60 exit 


See also Example 11-12 and Example 8-4. 
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8.4. Operator Precedence 


In a script, operations execute in order of precedence: the higher precedence operations execute before the 
lower precedence ones. [1] 


Table 8-1. Operator Precedence 


exponentiation arithmetic operation 
multiplication, division, 
modulo 


trinary operator 
assignment 
<<= »»- combination assignment 


|... [LOWEST PRECEDENCE 


In practice, all you really need to remember is the following: 


e The "My Dear Aunt Sally" mantra (multiply, divide, add, subtract) for the familiar arithmetic 
operations. 

* The compound logical operators, &&, ll, -a, and -o have low precedence. 

* The order of evaluation of equal-precedence operators is usually left-to-right. 


Now, let's utilize our knowledge of operator precedence to analyze a couple of lines from the 
/etc/init.d/functions file, as found in the Fedora Core Linux distro. 


i 
2 
3 
4 
3 
6 
j 
8 
9 
LO 
Li 
L2 
13 
L4 
L5 
L6 
Uy 


18 
19 
20 
2 
pu 
2 
24 
25 
26 
2) 
28 
29 
30 
3l 
32 
319) 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 


wh 


wh 


aie 


ile | =m YSranalmingt -a "Suez" e 0 l? clo 
This looks rather daunting at first glance. 
Separate the conditions: 

ile || sim USueewweduccoeg]/ -a UO: -oe © ip clo 


=- eiom == ^^ -—goumeaiigm 2= 


If variable "$remaining" is not zero length 


AND (-a) 
variable "Sretry" is greater-than zero 
then 
the [ expresion-within-condition-brackets ] returns success (0) 


and the while-loop executes an iteration. 


Hwelweute "üeemgellitiom 1" ancl VegmeBtucm 2" *v91oeitore == 

ANDing them. Why? Because the AND (-a) has a lower precedence 
than th n and -gt operators, 

and therefore gets evaluated *last*. 


HEH EE HH EH HE AE FE AE FE FE AE FE HE FE ee EE aa 


Ea 


= ee sessi ig ilem =a =z YS{INOLOCALAe—h" | g then 


Again, separate the conditions: 
| -2 /ete/sysconmtig/ ilem =a er USHUNONOCANESS-—JE"U | > then 
==—Comelslicwoim í ds (coimeltibigum Z———- 


If file "/etc/sysconfig/il8n" exists 


AND (-a) 
variable SNOLOCALE is zero length 
then 
the [ test-expresion-within-condition-brackets ] returns success 


and the commands following execute. 


As before, the AND (-a) gets evaluated *last* 
because it has the lowest precedence of the operators within 
the test brackets. 


Note: 

S(NOLOCALE:-) is a parameter expansion that seems redundant. 
But, if SNOLOCALE has not been declared, it gets set to *null*, 
in effect declaring it. 

This makes a difference in some contexts. 


bracketed sections. 


To avoid confusion or error in a complex sequence of test operators, break up the sequence into 


i si [| USwdL9 ge UEW2W -o UWSyilY -it Wisi a "Sfilename" ] 

2 4 Unclear what's going on here... 

3 

A si [I WS" see "S92" Jp II [I “vi =e Vv$wy2v J] ee II -e "Siilewmsme" |] 

5 4 Much better -- the condition tests are grouped in logical sections. 
Notes 


[1] Precedence, in this context, has approximately the same meaning as priority 
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Chapter 9. Another Look at Variables 


Used properly, variables can add power and flexibility to scripts. This requires learning their subtleties and 
nuances. 


9.1. Internal Variables 


Builtin variables: 

variables affecting bash script behavior 
SBASH 

The path to the Bash binary itself 


bash$ echo $BASH 
/bin/bash 


SBASH_ENV 

An environmental variable pointing to a Bash startup file to be read when a script is invoked 
SBASH SUBSHELL 

A variable indicating the subshell level. This is a new addition to Bash, version 3. 


See Example 21-1 for usage. 

SBASHPID 
Process ID of the current instance of Bash. This is not the same as the $$ variable, but it often gives 
the same result. 


bash4$ echo $$ 
JOD RE, 


bash4$ echo $BASHPID 
11015 


bash4$ ps ax | grep bash4 


11015 igi sy/ 2 R 0:00 bash4 
But ... 
1 #!/bin/bash4 
2 
3 echo ASIS outside of subshell = $$" 4 9602 
4 echo "NSBASH SUBSHELL outside of subshell = $BASH SUBSHELL" # 0 
5 echo "\SBASHPID outside of subshell = SBASHPID" # 9602 
6 
7 echo 
8 
9 ( echo “\S\S inside gi eubsmell = $$" # 9602 
LO echo "\SBASH SUBSHELL inside of subshell = $BASH SUBSHELL" # 1 
Li echo "\SBASHPID inside of subshell = $BASHPID" ) # 9603 


L2 # Note that $$ returns PID of parent process. 
SBASH VERSINFO[n] 
A 6-element array containing version information about the installed release of Bash. This is similar 
to SBASH. VERSION, below, but a bit more detailed. 


# Bash version info: 


d 

2 

3 tor m aa (0 43 2 9 4 5 

4 do 

5 echo "BASH VERSINFO[$n] = ${BASH VERSINFO[$n])" 
6 done 
7 
8 
E 
0 
B 


# BASH VERSINFO[0] = 3 # Major version no. 
# BASH VERSINFO[1] = 00 # Minor version no. 
10 4 BASH VERSINFO[2] = 14 # Patch level. 
11 4 BASH VERSINFO[3] = 1 # Build version. 


12 4 BASH VERSINFO[4] = release # Release status. 
13 4 BASH VERSINFO[5] i386-redhat-linux-gnu # Architecture 
14 # (same as SMACHTYPE). 
SBASH VERSION 
The version of Bash installed on the system 


bash$ echo $BASH VERSION 
3.2.25(1)-release 


tcsh$ echo $BASH VERSION 
BASH VERSION: Undefined variable. 


Checking $BASH, VERSION is a good method of determining which shell is running. $SHELL does 
not necessarily give the correct answer. 

$CDPATH 
A colon-separated list of search paths available to the cd command, similar in function to the SPATH 
variable for binaries. The $CDPATH variable may be set in the local ~/.bashrc file. 


bash$ cd bash-doc 
lasing Cele loaginaclocs No strea tile ow Cirectory 


bash$ CDPATH=/usr/share/doc 
bash$ ed bash-doc 
/usr/share/doc/bash-doc 


bash$ echo $PWD 
/usr/share/doc/bash-doc 


SDIRSTACK 
The top value in the directory stack [1] (affected by pushd and popd) 


This builtin variable corresponds to the dirs command, however dirs shows the entire contents of the 
directory stack. 

SEDITOR 

The default editor invoked by a script, usually vi or emacs. 


SEUID 


"effective" user ID number 
Identification number of whatever identity the current user has assumed, perhaps by means of su. 


D The SEUID is not necessarily the same as the $UID. 


SFUNCNAM 
Name of the current function 


Lj 


L xeweze (0) 

2 1 

2 echo "SFUNCNAME now executing." # xyz23 now executing. 

a} 

5 

6 xyz23 

7 

8 echo "FUNCNAME = SFUNCNAME" # FUNCNAME = 

9 # Null value outside a function. 


See also Example A-50. 
SGLOBIGNORE 


A list of filename patterns to be excluded from matching in globbing. 


SGROUPS 


Groups current user belongs to 


This is a listing (array) of the group id numbers for current user, as recorded in /etc/passwd and 
/etc/group. 


root# echo $GROUPS 
0 


root# echo ${GROUPS[1] } 
1l 


root# echo S(GROUPS[5]) 


6 

$HOME 

Home directory of the user, usually /home/username (see Example 10-7) 
SHOSTNAME 

The hostname command assigns the system host name at bootup in an init script. However, the 

gethostname () function sets the Bash internal variable SHOSTNAME. See also Example 10-7. 
SHOSTTYPE 

host type 


Like $MACHTYPE, identifies the system hardware. 


bash$ echo S$HOSTTYPE 
i686 


internal field separator 
This variable determines how Bash recognizes fields, or word boundaries, when it interprets character 


strings. 


SIFS defaults to whitespace (space, tab, and newline), but may be changed, for example, to parse a 
comma-separated data file. Note that $* uses the first character held in $IFS. See Example 5-1. 


bash$ echo "SIFS" 


(With SIFS set to default, a blank line displays.) 


bash$ echo "SIFS" | cat -vte 

AS 

$ 

(Show whitespace: here a single space, ^I [horizontal tab], 
and newline, and display "$" at end-of-line.) 


bash$ bash -c 'set w x y z; IFS=":-;"; echo "$*"' 
Wee BW Z 
(Read commands from string and assign any arguments to pos params.) 


D SIFS does not handle whitespace the same as it does other characters. 


Example 9-1. $IFS and whitespace 


PRPRPPPRPRPRPHE PB 
XO 0 1 O) O1 4 (0. I9 EB. O io 0 1 O) O1 4 CQ) I9 FS 


RITRO fS) 
ho [= = 


NN 
Ow Ww 


WWWNN DN P2 
nop [= tay Wey (o9) SX) cen) 


[99 
w 


CAIA TOTOE EO 65 
KOVO ATCO On x 


a 
pari 


iS, fey E WS ESO dex. Wes quy ss 
(ey Wek oo] Sa) copy Gal SS (60) cis 


C1 O1 O1 
[R9 [= 


on 
w 


ony (ex, Xen (Gub fon] (Sn) Gu «Gl 
I: c» © © —] oy On dE 


#!/bin/bash 
# ifs.sh 


varl="a+b+c" 
var2="d-e-f" 
Were $=", Im bU 


IFS=+ 
# The plus sign will be interpreted as a separator. 


echo $varl JEN 
echo $var2 # d-e-f 
echo $var3 W^ jui, a 
echo 

IFS-2"-" 


# The plus sign reverts to default interpretation. 
# The minus sign will be interpreted as a separator. 
echo $varl 4 atbtc 


echo $var2 #def 
echo $var3 i Goin aL 
echo 

IFS-"," 


# The comma will be interpreted as a separator. 
# The minus sign reverts to default interpretation. 


echo $varl # atbtc 
echo $var2 f d-e-f 
echo $var3 i Gj d at 
echo 

IFS-2" " 


# The space character will be interpreted as a separator. 
4 The comma reverts to default interpretation. 


echo $varl # atbtc 
echo $var2 # d-e-f 
echo $var3 dt Gi al 
# # 


# However 
# SIFS treats whitespace differently than other characters. 


output_args_one_per_line() 


{ 


for arg 
do 

Scho WY [ Seueg] V 
done 4 ^ 2 


echo Geha UNES WOW 
CeO sss " 


wees" m 19 € " 


output args one per line $var # output args one per line 'echo "a 


Embed within brackets, for your viewing pleasure. 


b 


ic 


of echos echo "IBESS:" 


WAL Neue eclB-nleyE(elB-grg # Same pattern as above, 
p QUE XS WP lem Sulosizaicuicnme "aV ior Ww 


82 Note "empty" brackets. 
83 The same thing happens with the "FS" field separator in awk. 


(Many thanks, Stéphane Chazelas, for clarification and above examples.) 


See also Example 16-41, Example 11-7, and Example 19-14 for instructive examples of using SIFS. 

$ IGNOREEOF 

Ignore EOF: how many end-of-files (control-D) the shell will ignore before logging out. 

$LC COLLATE 
Often set in the .bashrc or /etc/profile files, this variable controls collation order in filename 
expansion and pattern matching. If mishandled, LC. COLLATE can cause unexpected results in 


filename globbing. 


$^) As of version 2.05 of Bash, filename globbing no longer distinguishes between 
lowercase and uppercase letters in a character range between brackets. For example, Is 
[A-M]* would match both Filel.txt and filel.txt. To revert to the customary 
behavior of bracket matching, set LC COLLATE to C by an export 
LC COLLATE-C in /etc/profile and/or ~/.bashrc. 
$LC CTYPE 
This internal variable controls character interpretation in globbing and pattern matching. 
SLINENO 
This variable is the line number of the shell script in which this variable appears. It has significance 
only within the script in which it appears, and is chiefly useful for debugging purposes. 


1 4 *** BEGIN DEBUG BLOCK *** 

2 last cmd arg-$ . # Save it. 

E 

4 echo "At line number $LINENO, variable \"v1\" = $v1" 
5 


echo "Last command argument processed = $1ast cmd arg" 
(S qp 3055 ENDIDEBUCGTBTOCRK eee 


SMACHTYPI 
machine type 


Identifies the system hardware. 


SOLDPWD 


SOSTYPI 


bash$ echo $MACHTYPE 
i686 


Old working directory ("OLD-Print- Working-Directory", previous directory you were in). 


Lj 


operating system type 


bash$ echo $OSTYPE 
TENNUR 


SPATH 


SPI 


P 


Path to binaries, usually /usr/bin/, /usr/X11R6/bin/, /usr/local/bin, etc. 


When given a command, the shell automatically does a hash table search on the directories listed in 
the path for the executable. The path is stored in the environmental variable, $PATH, a list of 
directories, separated by colons. Normally, the system stores the $PATH definition in 
/etc/profile and/or -/.bashrc (see Appendix G). 


bash$ echo $PATH 

/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/sbin:/usr/sbin 
PATH-$ (PATH) : /opt/bin appends the /opt /bin directory to the current path. In a script, it 
may be expedient to temporarily add a directory to the path in this way. When the script exits, this 
restores the original SPATH (a child process, such as a script, may not change the environment of the 
parent process, the shell). 


8^; The current "working directory", . /, is usually omitted from the $PATH as a security 
measure. 


ESTATUS 


Array variable holding exit status(es) of last executed foreground pipe. 


bash$ echo $PIPESTATUS 


bash$ 1s -al | bogus command 
bash: bogus command: command not found 
bash$ echo $(PIPESTATUS[1]) 


bash$ ls -al | bogus command 

bash: bogus command: command not found 
bash$ echo $? 

LAY 


The members of the $PIPESTATUS array hold the exit status of each respective command executed 
in a pipe. SPIPESTATUS [0] holds the exit status of the first command in the pipe, 
SPIPESTATUS[1] the exit status of the second command, and so on. 


The SPIPESTATUS variable may contain an erroneous 0 value in a login shell (in 
releases prior to 3.0 of Bash). 


tcsh$ bash 
bash$ who | grep nobody | sort 


bash$ echo $(PIPESTATUS[*]) 
0 


The above lines contained in a script would produce the expected 0 1 0 output. 


Thank you, Wayne Pollock for pointing this out and supplying the above example. 


8^; The SPIPESTATUS variable gives unexpected results in some contexts. 


bash$ echo $BASH VERSION 
3.00.14(1)-release 


bash$ $ 1s | bogus command | wc 
bash: bogus, command: command not found 
0 0 0 


bash$ echo $(PIPESTATUS[Q]) 
141 127 0 


Chet Ramey attributes the above output to the behavior of Is. If /s writes to a pipe 
whose output is not read, then STGPIPE kills it, and its exit status is 141. Otherwise 
its exit status is 0, as expected. This likewise is the case for tr. 


8") SPIPESTATUS is a "volatile" variable. It needs to be captured immediately after the 
pipe in question, before any other command intervenes. 


bash$ $ 1s | bogus command | wc 
bash: bogus command: command not found 
0 0 0 


bash$ echo $(PIPESTATUS[Q]) 
O i127 @ 


bash$ echo $(PIPESTATUS[Q]) 
0 


$^) The pipefail option may be useful in cases where SPIPESTATUS does not give the 
desired information. 


SPPID 


The SPPID of a process is the process ID (pid) of its parent process. [2] 


Compare this with the pidof command. 
SPROMPT COMMAND 
A variable holding a command to be executed just before the primary prompt, $PS1 is to be 


displayed. 
$PS1 
This is the main prompt, seen at the command-line. 
$PS2 
The secondary prompt, seen when additional input is expected. It displays as ">". 
$PS3 
The tertiary prompt, displayed in a select loop (see Example 11-29). 
SPS4 
The quartenary prompt, shown at the beginning of each line of output when invoking a script with the 
-x option. It displays as "+". 
SPWD 


Working directory (directory you are in at the time) 
This is the analog to the pwd builtin command. 


!/bin/bash 


E_WRONG_DIRECTORY=83 


hs (Gs) [m5 [2 


5 clear # Clear screen. 

6 

7 TargetDirectory-/home/bozo/projects/GreatAmericanNovel 

8 

9 cd STargetDirectory 
l0 exeo "Dele: stale wiles inm StargeatDirettory 
TTL 
12 if [ "SPWD" != "STargetDirectory" ] 
13 then # Keep from wiping out wrong directory by accident. 
14 echo "Wrong directory!" 
i5 echo "In SPWD, rather than $TargetDirectory!" 
16 echo "Bailing out!" 
17 exit $E WRONG DIRECTORY 
Jg a 
19 
AQ i -e 5 
21 rm .[A-Za-z0-9]* # Delete dotfiles. 
22 ram He epu sake to remove filenames beginning with multiple dots. 
23) (SMODIE -6 Corgo imp =i 7) will also work. 
24 Lagi, SoCo LOM jxoximiE3umg) EMIS SUE. 
25 
26 A filename (^basename') may contain all characters in the 0 - 255 range, 
27 #+ except "/". 
28 Deleting files beginning with weird characters, such as - 
29 #+ is left as an exercise. 
30 
31 echo 
32 cehoa onc 
33 echo "Old files deleted in STargetDirectory." 
34 echo 
35 
36 # Various other operations here, as necessary. 
37 
38 exit S? 


SREPLY 
The default value when a variable is not supplied to read. Also applicable to select menus, but only 
supplies the item number of the variable chosen, not the value of the variable itself. 


1 #!/bin/bash 

2 # reply.sh 

3 

4 # REPLY is the default value for a 'read' command. 

3 

6 echo 

7 echo -n "What is your favorite vegetable? " 

8 read 

9 

10 echo "Your favorite vegetable is SREPLY." 

11 # REPLY holds the value of last "read" if and only if 
12 #+ no variable supplied. 

43 

14 echo 

15 echo =n "meg Sue: youi cevoplee dew V 

16 read fruit 

L echo "Neue reyot ii uade 3. Situwuls" 

re echo Subs oo 

19 sero “waluS oi wStüEIY xe nes SISuSEgb S 

20 # SREPLY is still set to its previous value because 
21 #+ the variable $fruit absorbed the new "read" value. 
BE 

23 echo 

24 

25 ew ( 


SSECONDS 


The number of seconds the script has been running. 
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#!/bin/bash 


TIME LIMIT-10 


INTERVAL-1 


echo 


Scho Uiit (OleuwEx(plL-XO ico exile losirore Siw 


echo 


while [ "SSECONDS" -le "STIME LIMIT" ] 


do 


aie [ WSSIEHCONJDDSV -eG i I 


then 


units=second 


else 


units=seconds 


f 


echo "This 


Script has been running $S] 


# On a slow or overburdened machine, 
#+ every once in a while. 


sleep SINT 
done 


ceho =e Wat 


exalic 0) 


S$SHELLOPTS 
The list of enabled shell options, a readonly variable. 


ERVAL 


# Beep! 


bash$ echo $SHELLOPTS 
braceexpand:hashall:histexpand:monitor:history:interactive-comments:emacs 


SSHLVL 


E LIMIT seconds." 


ECONDS $units." 
the script may skip a count 


Shell level, how deeply Bash is nested. [3] If, at the command-line, $SHLVL is 1, then in a script it 
will increment to 2. 


H This variable is not affected by subshells. Use SBASH SUBSHELL when you need 
. an indication of subshell nesting. 


STMOUT 


If the $TMOUT environmental variable is set to a non-zero value t ime, then the shell prompt will 
time out after $t ime seconds. This will cause a logout. 


As of version 2.05b of Bash, it is now possible to use $TMOUT in a script in combination with read. 


d 
2 
3 
4 
5 
6 
7 
8 


9 
JL) 
at 
12 
43 
14 


.05b and later. 


Seconds to answer!" 


# Works in scripts for Bash, versions 2 
TMOUT=3 # Prompt times out at three seconds. 
echo "What is your favorite song?" 
echo "Quickly now, you only have STMOUT 
read song 
3e | =z YSsome! | 
then 
song="(no answer)" 
# Default response. 
EL 


15 echo "Your favorite song is $song." 


There are other, more complex, ways of implementing timed input in a script. One alternative is to set 
up a timing loop to signal the script when it times out. This also requires a signal handling routine to 
trap (see Example 31-5) the interrupt generated by the timing loop (whew!). 


Example 9-2. Timed Input 


#!/bin/bash 
# timed-input.sh 


# TMOUT=3 Also works, as of newer versions of Bash. 


TIMER_INTERRUPT=14 
TIMELIMIT-3 # Three seconds in this instance. 
# May be set to different value. 


PrintAnswer () 
{ 
if [ "Sanswer" = TIMEOUT ] 
then 
echo Sanswer 
else # Don't want to mix up the two instances. 
echo "Your favorite veggie is Sanswer" 
kill $! 4 Kills no-longer-needed TimerOn function 
#+ running in background. 
# S! is PID of last job running in background. 


(se; -— fey) (Out des (63) [w3y [5 (03) wer er c iex; nl des tes) [soy [3 


= 


N 
O «o 
Fh 
H 


YPE [ES [esi 
(Ogp ges (93 [R5 [3 
— 


TimerOn() 


{ 


Ny NM 
Oo 


Sleep $TIMELIMIT && kill -s 14 $$ & 
# Waits 3 seconds, then sends sigalarm to script. 


w w [SS ISS) 
([— Sy Wey O9 


Ww 
N 


Intl14Vector() 

( 
answer-"TIMEOUT" 
PrintAnswer 
exit STIMER_INTERRUPT 


CO0 CO CO CO CO CO CO 
wO 0 -—-1 O| O1 BW 


trap Intl4Vector STIMER_INTERRUPT 
# Timer interrupt (14) subverted for our purposes. 


echo "What is your favorite vegetable " 
TimerOn 

read answer 

PrintAnswer 


Admittedly, this is a kludgy implementation of timed input. 
However, the "-t" option to "read" simplifies this task. 
See the "t-out.sh" script. 

However, what about timing not just single user input, 

but an entire script? 


BP PE Qe Ge Gam a gae am 
We) (oe c rex Gi ges (93. [S5 (3 «5» 


or O1 
(- & 


on 
N 
f 
t 


If you need something really elegant 
T (Cloyavsjilcleie 5 imgasslhceapmpil ceat omega C o Cirp 


‘Sm (Gal y 
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56 
5 
58 


#+ using appropriate library functions, such as 'alarm' and 'setitimer.' 


exit 0 


An alternative is using stty. 


Example 9-3. Once more, timed input 


(sy =] fey) (rl des (G9. A eS) ex Ger cp fey) (ap oS te) [soy [5 


#!/bin/bash 
# timeout.sh 


# Written by Stephane Chazelas, 
#+ and modified by the document author. 


INTERVAL=5 # timeout interval 


timedout read() { 
timeout-$1 
varname-$2 
ollel rry [etj SEEN c 
stty -icanon min 0 time ${timeout}0 
eval read $varname # or just read Svarname 
atty “Sioilcl rey sete inga” 
# See man page for "stty." 


echo; echo -n "What's your name? Quick! " 
timedout read SINTERVAL your name 


# This may not work on every terminal type. 
# The maximum timeout depends on the terminal. 
su (ue de orten 25.5 secos). 
echo 
ai | | —m VUSweus eue" ] # If name input before timeout 
then 
echo "Your name is $your name." 
else 
echo "Timed out." 
ical 
echo 
# The behavior of this script differs somewhat from "timed-input.sh." 


# At each keystroke, the counter resets. 


esac d) 


Perhaps the simplest method is using the -t option to read. 


Example 9-4. Timed read 


(exy (Gn dex (68) [5 [5 


#!/bin/bash 
i? t-out.sh 
# Inspired by a suggestion from "syngin seven" (thanks). 


TIMELIMIT-4 # 4 seconds 


SU] 


read -t STIMELIMIT variable <&1 


7 
8 
9 d SE 
10 # In this instance, "«&1" is nee 
11 # but unnecessary for Bash 3.x. 
12 
13 echo 
14 
iS) aie ff = Yeyen eN i] P dij a GHIL AL 
16 then 
17 echo "Timed out, variable still 
18 else 
ig echo "variable = Svariable" 
20) tea 
AI 
22 exec (0) 


eec ror Basi io uoo 2 ox 


unset." 


User ID number 


Current user's user identification number, as recorded in /etc/passwd 


This is the current user's real id, even if she has temporarily assumed another identity through su. 
SUID is a readonly variable, not subject to change from the command line or within a script, and is 


the counterpart to the id builtin. 


Example 9-5. Am I root? 


1 #!/bin/bash 
2 a Gü-3 —1 09/5 Snc Am I TOOK Oe Tao 
3 
4 ROOT_UID=0 # Root has SUID O0. 
3 
@ mie || VSUHUDU -e6 "SIEOQDU IUUD |) a MALL the real "seo" please stanci wo? 
7 then 
8 echo You are root. u 
9 else 
10 echo "You are just an ordinary user (but mom loves you just the same)." 
IL Ea 
12 
13 exit 0 
14 
IS) 
16 # # 
17 # Code below will not execute, because the script already exited. 
Te 
19 # An alternate method of getting to the root of matters: 
20 


21 ROOTUSER NAME-root 

22 

23 username= id -nu' ie (Olen eG username=  whoami- 
24 if [ "Susername" = "SROOTUSER NAME" ] 

25 then 

26 excelso) VROONEWV, TOGE, OSCE, WOU! are roor, Y 

27 else 

AS echo "You are just a regular fella." 

2O) se aL 


See also Example 2-3. 


pe) The variables SENV, SLOGNAME, SMAIL, $STERM, SUSER, and SUSERNAME are not 
Bash builtins. These are, however, often set as environmental variables in one of the 
Bash startup files. $SHELL, the name of the user's login shell, may be set from 
/etc/passwd or in an "init" script, and it is likewise not a Bash builtin. 


tcsh% echo $LOGNAME 
bozo 

tcsh$ echo $SHELL 
Mom eeeh 

tcsh% echo $TERM 
EXVE 


bash$ echo $LOGNAME 
bozo 

bash$ echo $SHELL 
/Ioxmv/AiE esm 

bash$ echo $TERM 
rxvt 


Positional Parameters 


$0, $1, $2, etc. 


St 


s* 


s 


Positional parameters, passed from command line to script, passed to a function, or set to a variable 
(see Example 4-5 and Example 15-16) 


Number of command-line arguments [4] or positional parameters (see Example 35-2) 


All of the positional parameters, seen as a single word 


E "S*" must be quoted. 


Same as $*, but each parameter is a quoted string, that is, the parameters are passed on intact, without 
interpretation or expansion. This means, among other things, that each parameter in the argument list 
is seen as a separate word. 


$^) Of course, "$@" should be quoted. 


Example 9-6. arglist: Listing arguments with $* and $@ 


!/bin/bash 
eueglstesE «elm 
Invoke this script with several arguments, such as "one two three". 


E BADARGS-65 
de [p E =m Visi ] 


echo "Usage: “basename $0! argumentl argument2 etc." 
exit $E BADARGS 


index-1 up Umabicalelllsixas CONE o 


PRPrPRP PP eR 
YNDOBWNFOCOMIDGABWNE 
Fh 
H 


echo Wivalsiestine; args wuspeloy WISSEN ve! 


18 
JS 
20 
21 
22 
23 
24 
25 
26 
2 
28 
29 
30 
Sab 
512 
33 
34 
35 
36 
3 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 


or iue; alin HSU p Doesnt work Dropar aie WEU sbenn i GWOTCEC, 
do 
echo "Arg #$index = Sarg" 
let "indext=1" 
done # $* sees all arguments as single word. 
echo "Entire arg list seen as single word." 


echo 
index=1 # Reset count. 

# What happens if you forget to do this? 
echo “iasting args with \"\S@\myt 


ror are im "Sem 
do 
echo "Arg #$index = Sarg" 
let "index+=1" 
done # S@ sees arguments as separate words. 
echo "Arg list seen as separate words." 


echo 
index=1 # Reset count. 
echo "Listing args with \$* (unquoted):" 


tor are iin SX 
do 
echo "Arg #$index = Sarg" 
let "indext=1" 
done # Unquoted $* sees arguments as separate words. 
echo "Arg list seen as separate words." 


ewe d) 


Following a shift, the $8 holds the remaining command-line parameters, lacking the previous $1, 
which was lost. 


(sey s (ex; (Gab des fes) [oy [0 


Ko) 


10 
LI 


#!/bin/bash 
# Invoke with ./scriptname 12 3 4 5 


echou S w à 2345 
shaft 

Gxelowg) Wise! w 2 3945 
Saal ETE 

echo US aN 4$ 345 


# Each "shift" loses parameter $1. 
# "S@" then contains the remaining parameters. 


The $@ special parameter finds use as a tool for filtering input into shell scripts. The cat "$9" 
construction accepts input to a script either from st din or from files given as parameters to the 


script. See Example 16-24 and Example 16-25. 


The $* and $@ parameters sometimes display inconsistent and puzzling behavior, 
depending on the setting of $IFS. 


Example 9-7. Inconsistent $* and $@ behavior 


#!/bin/bash 


# Erratic behavior of the "$*" and "S@" internal Bash variables, 
#+ depending on whether they are quoted or not. 


# Inconsistent handling of word splitting and linefeeds. 


set -- "First one" "second" 


# Setting the script arguments, 


echo 

echo 'IFS unchanged, using 
c=0 

moe 5 stig Ue 

Clo) «xelaror WS (esti) - ese gp 
done 

echo === 

echo 'IFS unchanged, using 
c=0 

itus aL Stig Se 

dio eo Se= Jem] 
done 

echon =< 

echo 'IFS unchanged, using 
c=0 

í£gi a ain Ue 

do echo "S((et=1)) 3 psa" 
done 

echo- = 

echo 'IFS unchanged, using 
c=0 

ioi 3 iim Se 

clo cxelmg. SCC DA § gau 
done 

Sele: c 

IFS-2: 

Cele Vanme SW. uepabsep Ww 
c=0 

Gre ak shia, Wise 

domechom Tiss tesedbn s [bose | a 
done 

echo c 

Seno VIN BW, spas Su 
c=0 

Om 4t sha. Se 

elo eco "St(er-i)»s [Sr 
done 

echo === 

var=$* 

eho USUS eU. qned: Tegel 
c=0 

itgue 3b aay. “Saverio! 

Glo celo. SS= S Ss]w 
done 

echo 

exelmo) "meo s. quee PaE 
c=0 

ROA SL gy SVENS 

do echo “"S((er=1))e Jes] 
done 

echot == 


Une laal ietol Orounve 
Sil, $25, 


"Sem 


# quoted 


# This line remains the same in every instance. 


# Echo args. 


ge 


# unquoted 


Ser 


se" 


(yes s) M 


(Gmene=sr>)) v 


[is 
ie) 


using Svar (var="$*")! 


1yps [exp 


using "Svar" 


1pps [say 


using "SQ"! 


we [gx] 


weing SEY 


ips (lesa) 


using $var (var-$Q)' 


iypyps iia 


(var-S$Q)' 


using VU Steel 


ipps pex] 


using essi UU 


ipps (Say 


using $var (var-"$Q")' 


1pps (ea 


up biy eals feXewesusi. wiel KSA owe. ASi Vs 


var="S*" 

SE UTSE, 
c=0 

itguc 3L Sua OVNA 
do echo "$ ((c+= 
done 

ehor ——— 

Scho Une gw 
c=0 

itoie 3b sua Uvade 
do echo "S((ct= 
done 

echo- = 

exelevoy "geo gu 
c=0 

fOr is 3 Weg" 
do echo "S$((ct= 
done 

echo- 

cho ES 
c=0 

moua ab alia Se 

do echo "S((ct= 
done 

Segre) ——— 

var=S@ 
celos menus, 
c=0 

itgue' 3L akin Shy 
do echo "S((ct= 
done 

echos — 

cehon "309 e Ug Ww 
c=0 

iue. 3b Shia, Skea! 
do echo "S$((ct= 
done 

echo- < 
var="S@" 

echon Vine SEU QW. 
c=0 

One 3L shia UNES 
do echo "S$ ((ct= 
done 

Sele, === 

Coo RBS eu 
c=0 

ir@ie 3k 309. Syeuc 
do echo "S((ct= 
done 

echo 

exaie (0) 

# This example 


script by Stephane Chazelas, 


(var="$*") ' 


(var= tS G) LU 


137 # and slightly modified by the document author. 


c The $@ and $* parameters differ only when between double quotes. 


Example 9-8. $* and $@ when SIFS is empty 


#!/bin/bash 


u^ 5i (Suse Sec Ut EOC, 
#+ then "S*" and "$Q" do not echo positional params as expected. 


mecho () # Echo positional parameters. 
{ 

Sein US. Sg SS 

} 


NPRPPP PPP PP 
SCOMIDGUTRWNHHEHOWCAIDGUARWNE 


JD esu Set, but empty. 

set abc Positional parameters. 

mcchoN SP aber 

# ^^ 

mecho $* a,b,c 

mecho $@ 2), loi, © 

Techo VEe yore 
2 
22 # The behavior of $* and $8 when SIFS is empty depends 
23 #+ on which Bash or sh version being run. 


24 # It is therefore inadvisable to depend on this "feature" in a script. 
25 

26 

27 4 Thanks, Stephane Chazelas. 

28 

29 exit 


Other Special Parameters 


S = 
Flags passed to script (using set). See Example 15-16. 


This was originally a ksh construct adopted into Bash, and unfortunately it does not 
seem to work reliably in Bash scripts. One possible use for it is to have a script 
self-test whether it is interactive. 


PID (process ID) of last job run in background 


LOG-$0.10g 

COMMAND1="sleep 100" 

exeo. VLeGCGime PUDS loxewelsepeoNUXel Comence roc fxescigoueg SOW >> "Supp" 
# So they can be monitored, and killed as necessary. 


echo >> "SLOG" 


# Logging commands. 


[M (x Wey (Ges | cxy Wal tes Gs) Iss) [2S 


PR 


echo =m "PID of \"SCOMMANDI\":; W >> "SLOG" 


$$ 


12 ${COMMAND1} & 

13 echo SI >> "SLOG" 

14 # PID of "sleep 100": 1506 

1.5 

16 4$ Thank you, Jacques Lederer, for suggesting this. 


Using $! for job control: 


| pēssibly hapging jos & 1 sleep STOUT} eval "Null 9 SU" e> /feew/ewlls } 
2 4 Forces completion of an ill-behaved program. 

3 4 Useful, for example, in init scripts. 
4 
5 


# Thank you, Sylvain Fourmanoit, for this creative use of the "!" variable. 
Or, alternately: 


1 # This example by Matthew Sage. 

2 4 Used with permission. 

3 

4 TIMEOUT-30 # Timeout value in seconds 

5 count=0 

6 

7 possibly hanging job & { 

8 while ((count « TIMEOUT )); do 

9 eval V 1 =l Uxwee/54)" | ee ((Coumnt = TIMMEOWE) )) V 
10 4 /proc is where information about running processes is found. 
Jd. # "-d" tests whether it exists (whether directory exists). 
12 # So, we're waiting for the job in question to show up. 
3,3 ( (Gower) J) 
14 sleep 1 
JUS done 
16 eval V sd "Aeeoc/SIV | @& kill -15 Si 
17 i? IIE cae hanmgkoac Joo ie sewueweshexey, IL ie. 
1g } 


Special variable set to final argument of previous command executed. 


Example 9-9. Underscore variable 


1 #!/bin/bash 
2 
3 eco S_ /bin/bash 
4 Just called /bin/bash to run the script. 
E Note that this will vary according to 
6 + how the script is invoked. 
7 
8 du »/dev/null So no output from command. 
9 echo S. du 
10 
11 Ys ell >/dev/nulll So no output from command. 
l2 echo $ -al (last argument) 
13 
l4 c 
i5 echo $9. 


Exit status of a command, function, or the script itself (see Example 24-7) 


Process ID (PID) of the script itself. [5] The $$ variable often finds use in scripts to construct 
"unique" temp file names (see Example 31-6, Example 16-31, and Example 15-27). This is usually 
simpler than invoking mktemp. 


Notes 


[1] A stack register is a set of consecutive memory locations, such that the values stored (pushed) are 
retrieved (popped) in reverse order. The last value stored is the first retrieved. This is sometimes called 
a LIFO (last-in-first-out) or pushdown stack. 


[2] The PID of the currently running script is $$, of course. 
[3] 


Somewhat analogous to recursion, in this context nesting refers to a pattern embedded within a larger 
pattern. One of the definitions of nest, according to the 1913 edition of Webster's Dictionary, illustrates 
this beautifully: "A collection of boxes, cases, or the like, of graduated size, each put within the one next 
larger." 


[4] The words "argument" and "parameter" are often used interchangeably. In the context of this document, 
they have the same precise meaning: a variable passed to a script or function. 


[5] Within a script, inside a subshell, $$ returns the PID of the script, not the subshell. 
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9.2. Typing variables: declare or typeset 


The declare or typeset builtins, which are exact synonyms, permit modifying the properties of variables. This 
is a very weak form of the typing [1] available in certain programming languages. The declare command is 
specific to version 2 or later of Bash. The typeset command also works in ksh scripts. 


declare/typeset options 


-r readonly 
(declare -r vari works the same as readonly var1) 


This is the rough equivalent of the C const type qualifier. An attempt to change the value of a 
readonly variable fails with an error message. 


1 declar r varl=1 

2 echo Vyevel = Sivas il # varl = 1 

3 

4 (( weld D) # x.sh: line 4: varl: readonly variable 
-i integer 

1 declar i number 

2 4 The script will treat subsequent occurrences of "number" as an integer. 

3 

4 number-3 

5 echo "Number = Snumber" # Number = 3 

6 

7 number=three 

8 echo "Number = Snumber" # Number = 0 


9 # Tries to evaluate the string "three" as an integer. 
Certain arithmetic operations are permitted for declared integer variables without the need for expr or 
let. 
1 n=6/3 
2 echo "n = $n" * n = 6/3 
3 
4 declare -i n 
5 n=6/3 
6 echo "n — $n" up dab e A 
-àarray 


1 declare -a indices 
The variable indices will be treated as an array. 
-f function(s) 


L eleeleuse -i 
A declare -f line with no arguments in a script causes a listing of all the functions previously 
defined in that script. 


1 declare -f function name 
A declare -f function name in a script lists just the function named. 
-x export 


JL eleclleuss -x Welles) 
This declares a variable as available for exporting outside the environment of the script itself. 
-x var=$value 


i deeler z yer s=3 1S) 
The declare command permits assigning a value to a variable in the same statement as setting its 
properties. 


Example 9-10. Using declare to type variables 


#!/bin/bash 


fonel ¢) 
{ 


Sehe Trig 15 ch 3CDUONCAE 3.539 c 


declare -f # Lists the function above. 

echo 

declare -i varl # varl is an integer. 

varl=2367 

echo "varl declared as $varl" 

varl-varl-*1l # Integer declaration eliminates the need for 'let'. 


echo "varl incremented by 1 is $varl." 

# Attempt to change variable declared as integer. 

echo "Attempting to change varl to floating point value, 2367.1." 
varl=2367.1 # Results in error message, with no change to variable. 
echo Vyar is grill  Syeucil s 


NPRPRPRPRP PPP PY 
O io 0» -1 O Oi i$ (0. M. I. O xo O0 -1 OY O1 e CQ) I S 


ZW 

22 echo 

23 

24 declar r var2-13.36 'declare' permits setting a variable property 
25 * and simultaneously assigning it a value. 

26 echo "var2 declared as $var2" Attempt to change readonly variable. 

27 var2=13.37 Generates error message, and exit from script. 
28 

29) echo Uyar 1s SELL SwaeD” This line will not execute. 

30 

Si c» (0) Script will not exit here. 


® Using the declare builtin restricts the scope of a variable. 


EOG K) 


FOO="bar" 


© = o Or & 69 NH r3 


foo 
9 echo SFOO 


2) oeus # Prints bar. 
However... 


feo (01 
declare FOO-"bar" 


8 echo $FOO 

gj 

10 

Lil osie jp Beints AOT 

17 

13 

14 4 Thank you, Michael Iatrou, for pointing this out. 


9.2.1. Another use for declare 


The declare command can be helpful in identifying variables, environmental or otherwise. This can be 
especially useful with arrays. 


bash$ declare | grep HOME 
HOME-/home/bozo 


bash$ zzy=68 
bash$ declare | grep zzy 
ZZy=68 


bash$ Colors=([0]="purple" [1]="reddish-orange" [2]="light green") 
bash$ echo ${Colors[@] } 

purple reddish-orange light green 

bash$ declare | grep Colors 

Colors=([0]="purple" [1]="reddish-orange" [2]="light green") 


Notes 


[1] In this context, typing a variable means to classify it and restrict its properties. For example, a variable 
declared or typed as an integer is no longer available for string operations. 


Clacilaies =i IDQNE 


intvar=23 

echo "Sintvar" # 23 
intvar=stringval 

echo "Sintvar" # 0 


Oy OO NOTES 


Prev Home Next 
Another Look at Variables Up $RANDOM: generate random 
integer 

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Chapter 9. Another Look at Variables Next 


9.3. $RANDOM: generate random integer 


SRANDOM is an internal Bash function (not a constant) that returns a pseudorandom [1] integer in the range 0 
- 32767. It should not be used to generate an encryption key. 


Example 9-11. Generating random numbers 


#!/bin/bash 


# SRANDOM returns a different random integer at each invocation. 
# Nominal range: 0 - 32767 (signed 16-bit integer). 


MAXCOUNT=10 
count=1 


echo 
echo "SMAXCOUNT random numbers:" 

Chg N g 
while [ "$count" -le SMAXCOUNT ] # Generate 10 (SMAXCOUNT) random integers. 
do 

number-$RANDOM 

echo $number 

Ler Wemminte a= dL Ge DgXeuctewusysus Combine c 

done 

Chom i 


NPRPPPRP PPP PY 
O (o 0» -1 O Ui i (). M I2. O xo 0 -1 O OI i QI 


# If you need a random int within a certain range, use the 'modulo' operator. 


21 # This returns the remainder of a division operation. 

22 

23 RANGE-500 

24 

25 echo 

26 

27 number=SRANDOM 

28 let "number %= SRANGE" 

29 s jug 

30 echo "Random number less than SRANGE ---  $number" 

EI 

32 echo 

33 

34 

35 

36 # If you need a random integer greater than a lower bound, 
37 #+ then set up a test to discard all numbers below that. 
38 

39 FLOOR-200 

40 

41 number-0 #initialize 

42 while [ "$number" -la S$FLOOR ] 

43 do 

44 number-S$RANDOM 

45 done 

46 echo "Random number greater than $FLOOR --- $number" 

47 echo 

48 

49 # Let's examine a simple alternative to the above loop, namely 
50 # let "number = SRANDOM + S$FLOOR" 

Syl # That would eliminate the while-loop and run faster. 
52 # But, there might be a problem with that. What is it? 


oul 
BS CO 


55 

56 # Combine above two techniques to retrieve random number between two limits. 
57 number=0 #initialize 

58 while [ "$number" -le SFLOOR ] 

59 Clo 

60 number=$RANDOM 

61 let "number %= SRANGE" # Scales Snumber down within SRANGE. 

62 done 

63 echo "Random number between SFLOOR and SRANG 
64 echo 

65 

66 

67 

68 # Generate binary choice, that is, "true" or "false" value. 

69 BINARY=2 

70 T=1 

71 number=SRANDOM 

UZ 

73 let "number $- SBINARY" 

74 # Note that let "number >>= 14" gives a better random distribution 
75 #+ (right shifts out everything except last binary digit). 

T7 sb | "Smwmwioes" Sete) SE I 

77 then 
78 echo "TRUE" 

79 else 

80 echo "FALSE" 

GL dea 

82 

83 echo 

84 

85 

86 # Generate a toss of the dice. 

87 SPOTS-6 # Modulo 6 gives range 0 - 5. 


EY 


[s 
| 
| 


= Sinise! 


88 # Incrementing by 1 gives desired range of 1 Gr 

89 # Thanks, Paulo Marcel Coelho Aragao, for the simplification. 
90 diel=0 

91 die2=0 


92 4 Would it be better to just set SPOTS=7 and not add 1? Why or why not? 
93 
94 4 Tosses each die separately, and so gives correct odds. 


95 
96 let "diel = SRANDOM $ SSPOTS +1" # Roll first one. 
97 let "die2 = SRANDOM $ SSPOTS +1" # Roll second one. 
98 # Which arithmetic operation, above, has greater precedenc 
9) ++ modulo: s) Or adcdaitbaon. (ie 
100 
101 


02 Mar "lmao wy = SCS sp SCEA 

103 echo "Throw of the dice = $throw" 
104 echo 

105 

106 

107 esac d) 


Example 9-12. Picking a random card from a deck 


1 #!/bin/bash 

2 4 pick-card.sh 

3 

4 # This is an example of choosing random elements of an array. 
5 

6 


# Pick a card, any card. 


Suites="Clubs 
Diamonds 
Hearts 
Spades" 


Denominations="2 


(oy a) (ox; (rl des (53) 


21 

2:2. ii) 

23 Jack 

24 Queen 
25 King 

26 Ace" 


28 # Note variables spread over multiple lines. 
31 suite-($Suites) # Read into array variable. 
32 denomination- ($Denominations) 


34 num_suites=S{#suite[*] } # Count how many elements. 
35 num_denominations=$ {#denomination[*] } 


37 echo -n "${denomination[$ ((RANDOM$num denominations))]) of " 


38 echo S{suite[$((RANDOM%num_suites) ) ] } 


41 4 Sbozo sh pick-cards.sh 
42 4 Jack of Clubs 


45 # Thank you, "jipe," for pointing out this use of S$RANDOM. 
46 exit 0 


Example 9-13. Brownian Motion Simulation 


!/bin/bash 

brownian.sh 

Author: Mendel Cooper 
Reldate: 10/26/07 
License: GPL3 


[This script models Brownian motion: 
+ the random wanderings of tiny particles in a fluid, 
as they are buffeted by random currents and collisions. 


+ This is colloquially known as the "Drunkard's Walk." 


It can also be considered as a stripped-down simulation of a 
* Galton Board, a slanted board with a pattern of pegs, 

+ down which rolls a succession of marbles, one at a time. 

+ At the bottom is a row of slots or catch basins in which 

+ the marbles come to rest at the end of their journey. 

Think of it as a kind of bare-bones Pachinko game. 

As you see by running the script, 
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+ board tilt-angle, 
* angles of impact, 


the script 


* most of the marbles cluster around the center slot. 

* This is consistent with the expected binomial distribution. 
As a Galton Board simulation, 
* disregards such parameters as 
rolling friction of the marbles, 

and elasticity of the pegs. 

To what extent does this affect the accuracy of the simulation? 


PASSES-500 Number of particle interactions / marbles. 

ROWS-10 Number of "collisions" (or horiz. peg rows). 

RANGE-3 0- 2 output range from SRANDOM. 

POS-0 IbSYES/IeaLS je, POEto. 

RANDOM-$$ Seeds the random number generator from PID 
tr Qt SCPI s 

declare -a Slots Array holding cumulative results of passes. 


NUMSLOTS-21 


lBastqeabellbabzer Sikes (0 4 
for i in $( seq S$NUMSLOTS ) 
do 


Number of slots at bottom of board. 


Zero out all elements of the array. 


Slots[$i]-0 

done 

echo # Blank line at beginning of run. 
} 

Show slots (x 4 

Schol ne en 

rom aL im S( See SNUMISIOMS ) # Pretty-print array elements. 

do 
joresinicie WS SrelY Siue] # Allot thr spaces per result. 

done 

echo # Row of slots: 

echo " | | | | | | | | | | | | | | | | | | | | 

echo " Qd 

echo 4$ Note that if the count within any particular slot exceeds 99, 

#+ it messes up the display. 
# Running only(!) 500 passes usually avoids this. 
} 

Move () { # Move one unit right / left, or stay put. 
Move=SRANDOM # How random is SRANDOM? Well, let's s 
let "Move %= RANGE" # Normalize into range of 0 - 2. 
case "SMove" in 

0 ) pF # Do nothing, i.e., stay in place. 
à ) ((208==))) 5 # Left. 
2- 3 T füsxolSsispL B # Right. 
"* j| xexelovo) Sin) Vümnewrowe UE 4 Anomaly! (Should never occur.) 
esac 
} 

Play () { # Single pass (inner loop). 

i=0 

wüscLle T WSs ike VSR ] # One event per row. 

do 
Move 
(Cisse) B 

done 


86 SHIFT-11 
897 ler EOS T= (SS 
ee (t SueuslieeosSsls« ) d 


+ 
= 


muy Ibi, ume] imei L0 2 
hift "zero position" to center. 
EBUG: echo SPOS 


+ 
ca 


iw) 


92 Run () { # Outer loop. 


94 ade | Ws" lt "SNSSmS" ] 


98 POS=0 # Reset to zero. Why? 


104 # main () 

LOS mpe ie aed bab e Silos 
106 Run 

LOV Show Sors 


110 esate SF 


# Exercises: 


Show the results in a vertical bar graph, or as an alternative, 
T a scattergram. 

# 2) Alter the script to use /dev/urandom instead of SRANDOM. 

# Will this make the results more random? 


PRPRPPRPRPP 
sl] py (On des 165 Je» (3 
+ 
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Jipe points out a set of techniques for generating random numbers within a range. 


il Generate random number between 6 and 30. 

2 rnumber-$ ( (RANDOM%25+6) ) 

3 

4 Generate random number in the same 6 - 30 range, 
5 #+ but the number must be evenly divisible by 3. 
6 rnumber=$ ( ( (RANDOM%30/3+1) *3) ) 

7 

8 Note that this will not work all the time. 

9 It fails if SRANDOM$30 returns O0. 

TO 

WI Frank Wang suggests the following alternative: 
IZ rnumber-$(( RANDOM%27/3%*3+6 )) 


Bill Gradwohl came up with an improved formula that works for positive numbers. 


1 rnumber-$ (((RANDOM$ (max-min*divisibleBy))/divisibleBy*divisibleBy-4min)) 
Here Bill presents a versatile function that returns a random number between two specified values. 


Example 9-14. Random between values 


#!/bin/bash 

# random-between.sh 

# Random number between two specified values. 

# Script by Bill Gradwohl, with minor modifications by the document author. 
# Used with permission. 


(eo) | (ox, Onl HSS. (o. [m3 [L5 


randomBetween() { 
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Generates a positive or negative random number 
+ between $min and $max 
+ and divisible by $divisibleBy. 
Gives a "reasonably random" distribution of return values. 


Bill Gradwoht — Oct 1, 2003 


syntax() { 
Function embedded within function. 
echo 
echo "Syntax: randomBetween [min] [max] [multiple]" 
echo 
echo -n "Expects up to 3 passed parameters, " 
echo "but all are completely optional." 
echo "min is the minimum value" 
echo "max is the maximum value" 
echo -n "multiple specifies that the answer must be " 
echo "a multiple of this value." 
echo " i.e. answer must be evenly divisible by this number." 
echo 
echo "If any value is missing, defaults area supplied as: 0 32767 
echo -n "Successful completion returns 0, " 
echo "unsuccessful completion returns" 
echo WEWINGIE Loin Sneek. cuml JL" 
cho -n "The answer is returned in the global variable " 
echo "randomBetweenAnswer" 
cho -n "Negative values for any passed parameter are " 
echo "handled correctly." 


local min=${1:-0} 

local max=${2:-32767} 

local divisibleBy-$(3:-1) 

Default values assigned, in case parameters not passed to function. 


Joe x 
local spread 


Let's make sure the divisibleBy value is positive. 
S{divisibleBy} -lr 0 ] && divisibleBy-$((0-divisibleBy)) 


Sanity check. 

if [ S# -gt 3 -o S${divisibleBy} -eq 0 -o S${min} -eq ${max} ]; then 
syntax 
return 1 

FA 


# See if the min and max are reversed. 
aie [ Simia -oe Simes) p Coen 

# Swap them. 

x=S {min} 

min-$ {max} 

max=$ {x} 


# If min is itself not evenly divisible by S$divisibleBy, 
#+ then fix the min to be within range. 
if [ $((min/divisibleBy*divisibleBy)) -ne $(min) ]; then 
sue dp Simeo Ihe (6) J|. XeloYewo! 
min-$ ((min/divisibleBy*divisibleBy)) 
else 
min=S$ ((((min/divisibleBy) +1) *divisibleBy) ) 
Hes 
ít 


# If max is itself not evenly divisible by S$divisibleBy, 


1" 


76 


[en 
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#+ then fix the max to be within range. 
if [ $((max/divisibleBy*divisibleBy)) -ne $(max) ]; then 
if [| S{max} -it O |]; then 
max-$ ( (((max/divisibleBy) -1) *divisibleBy) ) 
else 
max=S ( (max/divisibleBy*divisibleBy) ) 
ER 
fr 


Now, to do the real work. 


Note that to get a proper distribution for the end points, 
t the range of random values has to be allowed to go between 
+ 0 and abs (max-min)-*divisibleBy, not just abs (max-min)-l. 


rj 


he slight increase will produce the proper distribution for the 
+ end points. 


Changing the formula to use abs(max-min)-*1 will still produce 
+ correct answers, but the randomness of those answers is faulty in 


de 


+ is considerably lower than when the correct formula is used. 


that the number of times the end points (Smin and Smax) are returned 


spread-$ ( (max-min)) 
Omair Eshkenazi points out that this test is unnecessary, 
+ since max and min have already been switched around. 
S{spread} -1t 0 ] && spread-S((0-spread)) 
let spread+=divisibleBy 
randomBetweenAnswer-$ ( ( (RANDOM%spread) /divisibleBy*divisibleBy+min) ) 


return 0 


However, Paulo Marcel Coelho Aragao points out that 
+ when $max and $min are not divisible by SdivisibleBy, 
ar "Else omea ietal ils}. 


He suggests instead the following formula: 
rnumber = $(((RANDOM$ (max-—min+1)+min) /divisibleBy*divisibleBy) ) 


dE JE db db db od 


# Let's test the function. 
min--14 

max=20 

divisibleBy=3 


# Generate an array of expected answers and check to make sure we get 
#+ at least one of each answer if we loop long enough. 


declar a answer 
minimum-$ {min} 
maximum-$ (max) 
if [ $((minimum/divisibleBy*divisibleBy)) -ne $(minimum) ]; then 
zar [p SWpweneoquw) =e © Tg lex 
minimum=S ((minimum/divisibleBy*divisibleBy)) 
else 
minimum-$ ( ( ( (minimum/divisibleBy) +1) *divisibleBy) ) 
£3 
fesk 


# If max is itself not evenly divisible by S$divisibleBy, 
#+ then fix the max to be within range. 
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if [ $((maximum/divisibleBy*divisibleBy)) -ne $(maximum) ]; then 
aba [| Simes Iie (0) |) 9 ise 
maximum-$ ( ( ( (maximum/divisibleBy) -1) *divisibleBy) ) 
else 
maximum-$ ( (maximum/divisibleBy*divisibleBy) ) 
i3 


# We need to generate only positive array subscripts, 
#+ so we need a displacement that that will guarantee 
#+ positive results. 


disp-$ ((0-minimum)) 

for ((i=S{minimum}; i<=${maximum}; i-*-divisibleBy)); do 
answer [i+disp]=0 

done 


# Now loop a large number of times to see what we get. 
loopIt=1000 # The script author suggests 100000, 
#+ but that takes a good long while. 


for ((2=0p ace$(Jlecplt]pp mme co 


# Note that we are specifying min and max in reversed order here to 
#+ make the function correct for this case. 


randomBetween $(max) $(min) ${divisibleBy} 


# Report an error if an answer is unexpected. 


[ ${randomBetweenAnswer} -lt $(min) -o ${randomBetweenAnswer} -gt ${max} ] \ 
&& echo MIN or MAX error - $(randomBetweenAnswer]! 
[ $((randomBetweenAnswer%${divisibleBy})) -ne 0 ] \ 


&& echo DIVISIBLE BY error - ${randomBetweenAnswer}! 
# Store the answer away statistically. 


answer[randomBetweenAnswer-*disp]-$ ( (answer [randomBetweenAnswert+disp]+1) ) 
done 


# Let's check the results 


for ((i=S{minimum}; i<=${maximum}; it+=divisibleBy)); do 

[ ${answer[itdisplacement]} -eq 0 ] ^ 

&& echo "We never got an answer of $i." \ 

|| echo "S{i} occurred ${answer[i+displacement]} times." 
done 
eve d) 


Just how random is SRANDOM? The best way to test this is to write a script that tracks the distribution of 
"random" numbers generated by SRANDOM. Let's roll a SRANDOM die a few times . . . 


Example 9-15. Rolling a single die with RANDOM 


#!/bin/bash 
# How random is RANDOM? 


RANDOM-$$ # Reseed the random number generator using script process ID. 
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PIPS=6 # A die has 6 pips. 
MAXTHROWS=600 # Increase this if you have nothing better to do with your time. 
throw=0 # Throw count. 
ones=0 j7 Musi slinslice lines CONES CO ASO; 
twos=0 #+ since an uninitialized variable is null, not zero. 
threes=0 
fours=0 
fives=0 
sixes=0 
aaie iaesullic O) 
{ 
echo 
echo "ones = $ones" 
echo "twos = Stwos" 
echo "threes = S$threes" 
echo "fours = Sfours" 
echo "fives = Sfives" 
echo "Sixes = S$sixes" 
echo 
} 
update count () 
( 
Gels USdIU. ata 
0) tet “ones += I"; # Since die has no "zero", this corresponds to 1. 
dL CMT WO SERES aL pe And Tcigub. TE OGC 
2) depo beG|es ce. NE 
a) lew ours. FH es 
4) let "fives += 1";; 
5J iet ts ixes t= HR 
esac 
} 
echo 
while [ "$throw" -lt "SMAXTHROWS" ] 
do 
let "diel = RANDOM $ SPIPS" 
update count $diel 
Wet CENCOW t= 104 
done 
print_result 


exit 0 
The scores should distribute fairly evenly, 
With SMAXTHROWS at 600, all should cluster around 100, 
Keep in mind that RANDOM is a pseudorandom generator, 
+ and not a spectacularly good one at that. 
Randomness is a deep and complex subject. 
Sufficiently long "random" sequences may exhibit 
* chaotic and other "non-random" behavior. 
Exercise (easy): 
Rewrite this script to flip a coin 1000 times. 
Choices are "HEADS" and "TAILS". 


assuming RANDOM is fairly random. 
plus=oi- manwe T0! ie SO. 


As we have seen in the last example, it is best to reseed the RANDOM generator each time it is invoked. Using 
the same seed for RANDOM repeats the same series of numbers. [2] (This mirrors the behavior of the 
random () function in C.) 


Example 9-16. Reseeding RANDOM 


#!/bin/bash 
# seeding-random.sh: Seeding the RANDOM variable. 


MAXCOUNT=25 # How many numbers to generate. 


il 

2 

3 

4 

E 

6 random numbers () 

qi 

8 count=0 

S) malle | WSeounme” ie "SWBONCONUNNM ] 
ii) cle 

il number-$RANDOM 

2 echo. m Hemost Y 

3 let "count += 1" 

4 done 

5 

6 

7 

8 


) 


echo; echo 


19 RANDOM-1 # Setting RANDOM seeds the random number generator. 
20 random numbers 


22 echo; echo 


24 RANDOM-1 Same seed for RANDOM... 


# 
25 random_numbers # ...reproduces th xact same number series. 
26 # 
27 # When is it useful to duplicate a "random" number series? 
28 
29 echo; echo 
30 
31 RANDOM=2 # Trying again, but with a different seed... 
32 random_numbers # gives a different number series. 
33 
34 echo; echo 
35 
36 RANDOM-$$ seeds RANDOM from process id of script. 
97 It is also possible to seed RANDOM from 'time' or 'date' commands. 
38 


39 Getting fancy... 


40 SEED=$ (head -1 /dev/urandom | od -N 1 | awk '{ print $2 }') 

41 Pseudo-random output fetched 

42 #+ from /dev/urandom (system pseudo-random device-file), 

43 #+ then converted to line of printable (octal) numbers by "od", 
44 #+ finally "awk" retrieves just one number for SEED. 


45 RANDOM=SSEED 
46 random numbers 


48 echo; echo 


50 exit 0 


8^; The /dev/urandom pseudo-device file provides a method of generating much more "random" 
pseudorandom numbers than the $RANDOM variable. dd if=/dev/urandom of-targetfile 
bs-1 count-XX creates a file of well-scattered pseudorandom numbers. However, assigning these 


numbers to a variable in a script requires a workaround, such as filtering through od (as in above 
example, Example 16-14, and Example A-36), or even piping to md5sum (see Example 35-14). 


There are also other ways to generate pseudorandom numbers in a script. Awk provides a convenient 
means of doing this. 


Example 9-17. Pseudorandom numbers, using awk 


#!/bin/bash 
# random2.sh: Returns a pseudorandom number in the range 0 - 1. 
# Uses the awk rand() function. 


AWE SCR TEM | siainel())p joreimte geewnel() p 


# Command(s) / parameters passed to awk 
# Note that srand() reseeds awk's random number generator. 


echo -n "Random number between 0 and 1 = " 


echo | awk "SAWKSCRIPT" 
# What happens if you leave out the 'echo'? 
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exit 0 
Exercises: 

20 
2l 1) Using a loop construct, print out 10 different random numbers. 
22 (Hint: you must reseed the "srand()" function with a different seed 
23 #+ in each pass through the loop. What happens if you fail to do this?) 
24 
25 2) Using an integer multiplier as a scaling factor, generate random numbers 
26 ipu in the range between 10 and 100. 
AT 
28 3) Same as exercise #2, above, but generate random integers this time. 


The date command also lends itself to generating pseudorandom integer sequences. 
Notes 


[1] True "randomness," insofar as it exists at all, can only be found in certain incompletely understood 
natural phenomena, such as radioactive decay. Computers only simulate randomness, and 
computer-generated sequences of "random" numbers are therefore referred to as pseudorandom. 


[2] The seed of a computer-generated pseudorandom number series can be considered an identification 
label. For example, think of the pseudorandom series with a seed of 23 as Series #23. 


A property of a pseurandom number series is the length of the cycle before it starts repeating itself. A 
good pseurandom generator will produce series with very long cycles. 
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Chapter 10. Manipulating Variables 


10. 


1. Manipulating Strings 


Bash supports a surprising number of string manipulation operations. Unfortunately, these tools lack a unified 
focus. Some are a subset of parameter substitution, and others fall under the functionality of the UNIX expr 
command. This results in inconsistent command syntax and overlap of functionality, not to mention 
confusion. 


String Length 


${#string } 
expr length $string 


These are the equivalent of strlen() in C. 


expr "$string" : '.*' 


1 stringZ-abcABC123ABCabc 

2 

3 echo ${#stringZ} # ALS 
4 echo ‘expr length $stringZ^ # 15 
S ele espr Wost ringat: Usu # 15 


Example 10-1. Inserting a blank line between paragraphs in a text file 
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!/bin/bash 
paragraph-space.sh 
Ver. 2.0, Reldate 05Aug08 


Inserts a blank line between paragraphs of a single-spaced text fil 
Usage: $0 «FILENAME 


MINLEN-60 # May need to change this value. 
Assume lines shorter than $MINLEN characters ending in a period 
+ terminate a paragraph. S xercises at end of script. 


while read line For as many lines as the input file has... 
do 
echo site Output the line itself. 


len=${#line} 


ie [Ip Selen ihe "UONDUNLESNUS Be "SIUS" Se XI NSW NI 
then echo Add a blank line immediately 
£3 + after short line terminated by a period. 
done 
exit 


# Exercises: 


# EE erac deer ee 

# 1) The script usually inserts a blank line at the end 

#+ Qut the target file. Piz this, 

# 2) Line 17 only considers periods as sentence terminators. 

# Modify this to include other common end-of-sentence characters, 
#+ sucha as ?, lp eme Y, 


Length of Matching Substring at Beginning of String 


expr match "$string" '$substring' 


Ssubstringis a regular expression. 


expr "$string" : '$substring' 
$substringisaregular expression. 


stringZ=abcABC123ABCabc 
# lier | 
# 12345678 


scho enpr matcha “Seieiiing ZY "alloys pA] VU 
echo empr “Seiciening~™” 3 aloe [pA] 55.29 


(ex, (mt dE (ES) [m5 [3 


# 8 
# 8 
Index 


expr index $string $substring 
Numerical position in $string of first character in $substring that matches. 


1 stringZ=abcABC123ABCabc 


2 # 123456 

53 edho expr abesse ost ame AM (QULA # 6 

4 if C Teo s 
5 

6 echo “expr index "$stringZ" lei # 3 

7 4 'c' (in #3 position) matches before '1'. 


This is the near equivalent of strchr() in C. 
Substring Extraction 


${string:position } 
Extracts substring from $stringat Sposition. 


If the Sst ring parameter is "*" or "@", then this extracts the positional parameters, [1] starting at 
Sposition. 

${string:position:length} 
Extracts $length characters of substring from $string at Sposition. 


1 stringZ=abcABC123ABCabc 
2 t OIZSASCSIED NN 
3 4 0-based indexing. 
4 
5 echo ${stringZ:0} # abcABC123ABCabc 
6 echo $ strerizngz:1) # bcABC123ABCabc 
T exelwo Sissi) # 23ABCabc 
8 
9 echo S(stringZ:7:3) i Q3 
10 # Three characters of substring. 
Li 
T2 
13 
14 # Is it possible to index from the right end of the string? 
15 
16 echo $[stringZ:-4) # abcABC123ABCabc 
17 4 Defaults to full string, as in ${parameter:-default}. 
18 4 However 
19 
20 echo S{stringZ: A) } # Cabc 
21 echo S{stringZ: -4) # Cabc 
22 # Now, it works. 
23 # Parentheses or added space "escape" the position parameter. 


24 


25 # Thank you, Dan Jacobson, for pointing this out. 
The position and length arguments can be "parameterized," that 1s, represented as a variable, rather 
than as a numerical constant. 


Example 10-2. Generating an 8-character "random" string 


1 #!/bin/bash 
2 wp TGUACI SIE Lins), lo 
3 4 Generating an 8-character "random" string. 
4 
5 if [ -n "S1" ] # If command-line argument present, 
6 then #+ then set start-string to it. 
7 str0-"$1" 
8 else # Else use PID of script as start-string. 
9 str0-"$$" 
JL) ita 
alle 
12. POS=2 47 Sterio Prom INOS A aig CHhe SELLUN. 
13 LEN-8 # Extract eight characters. 
14 
1S strl=S( echo "Se" | meber | xucleuws ) 
AKG Doubly scramble: SEEN REAR NS 
17 
18 randstring-"$(strl:$POS:$LEN)]" 
19 Can parameterize AAA Aana 
20 
21 echo "Srandstring" 
27 
23 ecu Se 
24 
25 4 bozo$ ./rand-string.sh my-password 
26 # 1bdd88c4 
A 
28 # No, this is is not recommended 
29 #+ as a method of generating hack-proof passwords. 


If the $string parameter is "*" or "@", then this extracts a maximum of $1ength positional 
parameters, starting at $5position. 


il eee 992) # Echoes second and following positional parameters. 

2 eero gia # Same as above. 

3 

4 echo s (02:09) # Echoes three positional parameters, starting at second. 


expr substr $string $position $length 
Extracts $length characters from $string starting at 5position. 


1 stringZ=abcABC123ABCabc 

2 t 1,2:9)215(5 7819) scan ce 

3 4 1-based indexing. 

4 

5 echo "ego substre Suisumem I 2° # ab 
6 echo “expr substr SstringZ 4 3' # ABC 


expr match "$string" '\($substring\)' 

Extracts Ssubst ring at beginning of $string, where Ssubst ring is a regular expression. 
expr "$string" : '\($substring\)' 

Extracts Ssubst ring at beginning of $string, where Ssubst ring is a regular expression. 


1 stringZ-abcABC123ABCabc 

2 # ======= 

3 

4| beho egor march “SSiciening "Wa [eee p oo 0-9] 3) V^ 
S. echo egr Vegri ronca UN S [nee DANI sw [O9] Ny "> 

(5; Sela, "(epus WSsicwaling~ g UX( oos as NS 


7 4 All of the above forms give an identical result. 
expr match "$string" '.*\($substring\)' 


# abcABCl 
# abcABCl 
# abcABCl 


Extracts Ssubstring at end of $string, where $substringis a regular expression. 


expr "$string" : '.*\($substring\)' 


Extracts Ssubstring at end of $string, where Ssubst ring is a regular expression. 


1 stringZ=abcABC123ABCabc 

2 # ====== 

3 

Al echo epr macch "Sees gu Y SN CDS [2€ 1 ec] [ee] 9 39 V* 
5 eco "ergo VESTEA mo Uu Xd s eco og Wwe 


Substring Removal 


${string#substring } 

Deletes shortest match of Ssubst ring from front of Sst ring. 
${string##substring } 

Deletes longest match of Ssubst ring from front of $string. 


stringZ=abcABC123ABCabc 
|---| shortest 
[SRE | longest 


1 

2 

3 

4 

5 echo ${stringZ#a*C} # 123ABCabc 

6 Strip out shortest match between 'a' and 'C'. 
7 

8 


echo ${stringZ##a*C} # abc 
g Strip out longest match between 'a' and 'C'. 


${string%substring } 
Deletes shortest match of S$subst ring from back of Sst ring. 


For example: 


# Rename all filenames in SPWD with "TXT" suffix to a 


# For example, "filel.TXI" becomes "filel.txt" 


il 

2 

3 

4 SUFF-TXT 
5 suff-txt 
6 
7 
8 


ioe SL sia SCs wa SSUM) 
do 
9 my =e Sal fassus: Sse 


# ABCabc 
# ABCabc 


SWE E 1H, 


10 # Leave unchanged everything *except* the shortest pattern match 


4.3 #+ starting from the right-hand-side of the variable $i 
if desired. 


12 done 444 This could be condensed into a "one-liner" 


14 # Thank you, Rory Winston. 
${string% %substring } 
Deletes longest match of Ssubst ring from back of Sst ring. 


1 stringZ=abcABC123ABCabc 

2 $ | | shortest 
3. ii Vee ea ae ea | longest 
4 


9 
6 
j 
8 
9 


echo ${stringZ%b*c} 
# Strip out shortest match between 


echo ${stringZ%%b*c} # a 
# Strip out longest match between 


This operator is useful for generating filenames. 


# abcABC123ABCa 


Vig! amei "VL. iie oek Oi Siam s 


I9" amel Viel, Erom back (ur Sentia. 


Example 10-3. Converting graphic file formats, with filename change 


Mop p opp ppHppÀG:! 
O (o 0» -1 O Oi i$ (). NM P. O xo 0 -1 O O1 4 CQ I E 


NNNNNN LPS 
=I) copy Gal ges (Gs) [sy qp 


N 
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CO CO CO CO CO CO CO CO CO Dd 
(eo <I) ey Gal Es. (39v [Sp [Lv (e» Wo 


3 


#!/bin/bash 
"ur OVES SNS 
# Converts 


# Uses the 
which is 


all the MacPaint image files in a directory to "pbm" 


"macptopbm" binary from the "netpbm" package, 
maintained by Brian Henderson 


(bryanh@giraffe-data.com). 


# Netpbm is a standard part of most Linux distros. 


ERATION=macptopbm 
SUFFIX-pbm 


# New filename 


wie | 
then 

directory=$1 
else 


Welw ] 


=h 


[SRUTIE IE ILS, 


# If directory name given as a script argument... 


directory=SPWD # Otherwis 


iE aL 


us 


# Assumes all files in the target 


current working directory. 


directory are MacPaint image files, 


Filename globbing. 


suffix off filename 
matches everything 
Ve",  abpvellhmsawes) c 


Redirect conversion to new filename. 
Delete original files after converting. 
Log what is happening to stdout. 


this script converts *all* the files in the current 


#+ with a ".mac" filename suffix. 
for file in $directory/* 
do 
filename-$[(file$.*c) Sesto "ime 
+ (ee 
#+ between '.' and 
SOPERATION $file > "Sfilename.$SUFFIX" 
rm -f $file 
echo "Sfilename.$SUFFIX" 
done 
lxxi (0) 
# Exercise: 
# "et d bs oak Jod os D de: 
# As it stands, 
#+ working directory. 


io Mochy 3d qt Work “omy om eiles wiren e Voie XE. 


Example 10-4. Converting streaming audio files to ogg 


Woy 109) «| (ox, (Un. HSS (63. ISy [55 


!/bin/bash 
ra2ogg.sh: 
Uses the 


the 
http://www.xiph.org/ 


Uses 


This 


Convert streaming audio files 


script may need appropriate codecs installed, 


(eo ieal) iO: OX 


"mplayer" media player program: 
http://www.mplayerhq.hu/homepage 
Vege 3ueuceucy cuo! "exse" E 


Suc’ AS iste 


format. 


# Possibly also the compat-libstdct+ package. 


OFILEPREF=S {1%%ra} W^ Sus (Quei ile. Wiese Simei ust, 
OFILESUFF-wav # Suffix for wav file. 
ODTETLES"SOPILEPEBP"U"SORTLESUBERM 


aie 


E NOARGS-85 


| = VERS | # Must specify a filename to convert. 


then 


Tal 


echo "Usage: “basename $0' [filename]" 
exit $E NOARGS 


HEH HE HT EH HH FE FE AE EE aE a Ee aE EE ae EE aE 


mplayer "$1" -ao pom:file-$OUTFILE 


oggenc "SOUTFILE" # Correct file extension automatically added by oggenc. 
HHH EH EH EHH THE EH EH FE FE HE FE HEE HEE HEH HE HHH EE HE EE EEE EE EE E E E E E E RE 


icu “SOQUEL # Delete intermediate *.wav file. 
# If you want to keep it, comment out above line. 
exit $? 
Note 


On a Website, simply clicking on a *.ram streaming audio file 
usually only downloads the URL of the actual *.ra audio file. 
You can then use "wget" or something similar 

to download the *.ra file itself. 


Exercises: 
AS US, daie GewISE Converte Gu ize tilememes 
Add flexibility by permitting use of *.ram and other filenames. 


If you're really ambitious, expand the script 

to do automatic downloads and conversions of streaming audio files. 
Given a URL, batch download streaming audio files (using "wget") 
and convert them on the fly. 


A simple emulation of getopt using substring-extraction constructs. 


Example 10-5. Emulating getopt 


#!/bin/bash 

# getopt-simple.sh 

# Author: Chris Morgan 

# Used in the ABS Guide with permission. 


getopt simple() 


{ 


echo "getopt_simple()" 
echo Vearameters are VEU 


tawai [ c: WSIW ] 

do 
echo "Processing parameter of: '$1'" 
sd [| S$queUsis = V/V j 
then 


16 tmp=${1:1} p Gtrup oft leading "/" 

L7 parameter=S {tmp%%=* } # Extract name. 

18 value=S$ {tmp##*=} # Extract value. 

L3 echo "Parameter: 'Sparameter', value: '$value'" 

20 eval $parameter-$value 

2 PA 

22 shaft 

23 done 

24 } 

25 

26 # Pass all options to getopt_simple(). 

27 getopt_simple $* 

28 

29) echo Vies ig "S egi" 

30 eeno "ies? as "USE" 

st 

32 exit 0 4 See also, UseGetOpt.sh, a modified versio of this script. 

33 

34 --—- 

35 

36 sh getopt example.sh /test-valuel /test2-value2 

aT 

38 Parameters are '/test-valuel /test2-value2' 

39 Processing parameter of: '/test-valuel' 

40 Parameter: 'test', value: 'valuel' 

41 Processing parameter of: '/test2-value2' 

42 Parameter: 'test2', value: 'value2' 

43 test is 'valuel' 

44 test2 is 'value2' 

45 
Substring Replacement 
${string/substring/replacement} 

Replace first match of Ssubst ring with Sreplacement. [2] 
${string//substring/replacement } 
Replace all matches of Ssubst ring with Sreplacement. 

1 stringZ=abcABC123ABCabc 

2 

3 echo ${stringZ/abc/xyz} xyZABC123ABCabc 

4 Replaces first match of 'abc' with 'xyz'. 

5 

6 echo ${stringZ//abc/xyz} XyzABC123ABCxyz 

7 Replaces all matches of 'abc' with # 'xyz'. 

8 

9 echo 

10) ceho VSTi neyt abcABC123ABCabc 

11 echo 

12 The string itself is not altered! 

13 

14 # Can the match and replacement strings be parameterized? 

15 match-abc 

16 repl-000 

17 echo ${stringZ/Smatch/$repl} # 000ABC123ABCabc 

18 4 i È 2E 

19 echo $(stringZ//$match/$repl) # 000ABC123ABCOO00 

20 4 Yes! 5s i uet sies 

AL 

22 echo 

25 

24 # What happens if no $replacement string is supplied? 

25 echo ${stringZ/abc} # ABCI23ABCabc 


26 echo $(stringZ//abc) # ABC123ABC 
27 # A simple deletion takes place. 


${string/#substring/replacement} 

If Ssubst ring matches front end of $string, substitute Sreplacement for $substring. 
${string/%substring/replacement } 

If Ssubst ring matches back end of Sst ring, substitute Sreplacement for Ssubstring. 


1 stringZ=abcABC123ABCabc 


2 

3 echo ${stringZ/#abc/XYZ} 4 XYZABC123ABCabc 

4 # Replaces front-end match of 'abc' with 'XYZ'. 
5 

6 echo $(stringZ/$abc/XYZ) # abcABC123ABCXYZ 

7 # Replaces back-end match of 'abc' with 'XYZ'. 


10.1.1. Manipulating strings using awk 


A Bash script may invoke the string manipulation facilities of awk as an alternative to using its built-in 
operations. 


Example 10-6. Alternate ways of extracting and locating substrings 


!/bin/bash 
substring-extraction.sh 


String=23skidool 
012345678 Bash 
123456789 awk 
Note different string indexing system: 
Bash numbers first character of string as 0. 
Awk numbers first character of string as 1. 


echo ${String:2:4} # position 3 (0-1-2), 4 characters long 
# skid 


# The awk equivalent of S${string:pos:length} is substr(string,pos,length) . 
echo | awk ' 

{ preine Stoste (YS Sx "V, S. 2) # skid 

} 


# Piping an empty "echo" to awk gives it dummy input, 
#+ and thus makes it unnecessary to supply a filename. 


NPRPRPP RPP PP PY 
COMIKDGTHRWNHFOWOMIRDUGUBRWNHE 


N N 
N he 


@eElao WW 


NO ON 
B W 


# And likewise: 


N 
[91] 


echo | awk ' 
(b Joweaumig. ande (UU UST eR UU W.  Wepesraluy 
} 


y # The awk equivalent of "expr index" 


ND PN 
=| xe 


# 
# 


N 
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3 
(GILG cueste SE joOsmica@im 3) 


U Ww D 
i= & wel 


exit 0 


10.1.2. Further Reference 


For more on string manipulation in scripts, refer to Section 10.2 and the relevant section of the expr command 


listing. 
Script examples: 


1. Example 16-9 

2. Example 10-9 

3. Example 10-10 
4. Example 10-11 
5. Example 10-13 
6. Example A-36 
7. Example A-41 


Notes 


[1] This applies to either command-line arguments or parameters passed to a function. 


[21] Note that $substringand $replacement may refer to either literal strings or variables, 
depending on context. See the first usage example. 


Prev Home Next 
SRANDOM: generate random Up Parameter Substitution 
integer 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Next 


Prev Chapter 10. Manipulating Variables 


10.2. Parameter Substitution 


Manipulating and/or expanding variables 


${parameter} 
Same as Sparameter, i.e., value of the variable parameter. In certain contexts, only the less 
ambiguous $ {parameter} form works. 


May be used for concatenating variables with strings. 


your_id=$ {USER}-on-$ (HOSTNAMI 
Scho "eeu nel! 

# 
echo "Old \$PATH = $PATH" 

PATH-S$(PATH):/opt/bin # Add /opt/bin to $PATH for duration of script. 
6 echo "New \SPATH = SPATH" 


$(parameter-default),$(parameter:-default]) 
If parameter not set, use default. 


ES 
— 


(Gri gm (GS) [eor fS 


1 varl=1 

2 var2-2 

3 # var3 is unset. 

4 

5 echo $(varili-$var2) # 1 

6 echo S(var3-$var2)] # 2 

353 A Note the $ prefix. 
8 

9 

10 

11 echo $[(username-'whoami']) 

12 4: HEMOSS icine cesi or woan , sic yariabls Suüsesrceane ae siewilil wiasienc . 


$^ S{parameter-default} and S(parameter:-default } are almost 
equivalent. The extra : makes a difference only when parameter has been declared, 
but is null. 


#!/bin/bash 
# param-sub.sh 


# Whether a variable has been declared 
#+ affects triggering of the default option 
#+ even if the variable is null. 


username0- 

echo "usernameO has been declared, but is set to null." 
echo "username0 = $(username0-'whoami'])" 

# Will not echo. 


ecno 


echo usernamel has not been declared. 
echo "usernamel = $(usernamel-'whoami']" 
# Will echo. 


(coy «| py (m dev GS IS) [= GS) Wer es) | fx) (On ges Gel IS) 45 


19 username2- 
20 echo "username2 has been declared, but is set to null." 


21 echo "username2 = $(username2:-'whoami')" 
22 t A 
23 # Will echo because of :- rather than just - in condition test. 


24 4 Compare to first instance, above. 


25 
26 
2 
28 
29 
30 
3 
32 
33 
34 
33 
36 
37 
38 
3 
40 
41 
42 
43 


# Once again: 


variable- 
# variable has been declared, but is set to null. 


echo "S{variable-0}" # (no output) 
echo "S{variable:-1}" od 

# ^ 

unset variable 

echo "S{variable-2}" s o2 

echo "S{variable:-3}" # 3 


esac 0 


The default parameter construct finds use in providing "missing" command-line arguments in scripts. 


|bombpmpmowmn 
Oi 4$ (Q0. NN. IB. O (o 00 -1 O) O1 iS CQ). IN) S 
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DEFAULT FILENAME-generic.data 

filename-$[(1:-$DEFAULT FILENAME] 

# If not otherwise specified, the following command block operates 
jones tuse Wevehovewesle’ 5 dartan 

#  Begin-Command-Block 


Se ESE 


End-Command-Block 


# From "hanoi2.bash" example: 

DISKS-$(1:-E NOPARAM) # Must specify how many disks. 
# Set SDISKS to $1 command-line-parameter, 

#+ or to $E NOPARAM if that is unset. 


See also Example 3-4, Example 30-2, and Example A-6. 


Compare this method with using an and list to supply a default command-line argument. 
${parameter=default}, ${parameter :=default} 


If parameter not set, set it to default. 


Both forms nearly equivalent. The : makes a difference only when $parameter has been declared 
and is null, [1] as above. 


JL 
2 
3 


echo $[var-abc) # abc 
echo ${var=xyz} # abc 
# Svar had already been set to abc, so it did not change. 


${parametert+talt_value}, ${parameter:+alt_value} 
If parameter set, use alt value, else use null string. 


Both forms nearly equivalent. The : makes a difference only when parameter has been declared 
and is null, see below. 


(ni dE (G8) ISS) p 


echo "###### \S{parametertalt_value} ########" 
echo 


a=S{paraml+xyz} 
echo "a = $a" # a= 


param2- 
a-$(param2*xyz)] 
echo "a = $a" # a = xyz 


param3=123 
a-$(param3*xyz) 
echo "a = $a" # a = xyz 


echo 
echo "###### NS(parameter:*alt value) ######HH" 
echo 


a=S{param4:+xyz} 
echo "a = Sa" # a= 


Mop pop opp PPP PY 
O (o 0 -1 O OQ! i£ () M P O «o 0 -1 0 


NON 
woe 


param5= 
a=S{param5:+xyz} 

echo "a = Sa" da 
# Different result from a=S{param5+xyz} 


NON 
dE w 
ll 


NNN 
sp xy Cal 


param6-123 
a=${param6:+xyz} 
29 echo "a = Sa" # a = xyz 
${parameter?err_msg}, ${parameter:?err_msg} 
If parameter set, use it, else print err_msg. 


N 
[99] 


Both forms nearly equivalent. The : makes a difference only when parameter has been declared 
and is null, as above. 


Example 10-7. Using parameter substitution and error messages 


#!/bin/bash 


Check some of the system's environmental variables. 
This is good preventative maintenanc 
i 
ie 


f, for example, SUSER, the name of the person at the console, is not set, 


Se SE ch OSE 


he machine will not recognize you. 


1 
2 
3 
4 
5 
6 
E 
8 S{HOSTNAME?} $(USER?) ${HOME?} S{MAIL?} 
9 echo 
10 echo "Name of the machine is SHOSTNAME. 
il 
Z 
3 
4 
5 
6 
7 
8 


EJ 


echo "You are SUSER." 

echo "Your home directory is $HOME." 
echo "Your mail INBOX is located in $MAIL." 

echo 

echo "If you are reading this message," 

echo "critical environmental variables have been set." 


echo 
il echo 

19 

20 

Zl 

du The $(variablename?) construction can also check 
23 #+ for variables set within the script. 

24 

25 ThisVariable-Value-of-ThisVariable 

26 Note, by the way, that string variables may be set 
27 #+ to characters disallowed in their names. 


28 : $(ThisVariable?) 
2,9) peho "weder. Wests Werelalolle 1S Sas syereieloley c 


31 echo; echo 


33 

SAL SEO SABR Uo 2 AB has not been sec. Ih 

S5 Since ZZXy23AB has not been set, 

36 #+ then the script terminates with an error message. 

317) 

38 You can specify th rror messag 

39 : S{variablename?"ERROR MESSAGE") 

40 

41 

42 Same result with: dummy variable-$(ZZXy23AB?)] 

43 dummy variable-$(ZZXy23AB?"ZXy23AB has not been set.") 
44 

45 echo $(ZZXy23AB?) >/dev/null 

46 

47 Compare these methods of checking whether a variable has been set 
48 #+ with "set -u" 

49 

50 

51 


52 echo "You will not see this message, because script already terminated." 


54 HERE=0 
55 exit SHERE # Will NOT exit here. 
56 


57 ; Im fact, tnis geript will recura an exit status lecho S7) oe 1. 


Example 10-8. Parameter substitution and "usage" messages 


#!/bin/bash 
# usage-message.sh 


il 
2 
3 
4 $(1?"Usage: $0 ARGUMENT") 

5 # Script exits here if command-line parameter absent, 
6 

Ji 

8 


#+ with following error message. 
# usage-message.sh: 1: Usage: usage-message.sh ARGUMENT 


9 echo "These two lines echo only if command-line parameter given." 


10 echo "command-line parameter = \"$1\"" 

Li 

12 exit 0 4 Will exit here only if command-line parameter present. 
13 


14 # Check the exit status, both with and without command-line parameter. 
15 # If command-line parameter present, then "$?" is 0. 
LE 4» iE not, chem VSP! 3e JL. 


Parameter substitution and/or expansion. The following expressions are the complement to the match in 
expr string operations (see Example 16-9). These particular ones are used mostly in parsing file path names. 


Variable length / Substring removal 


${#var} 
String length (number of characters in $var). For an array, ${#array} is the length of the first 
element in the array. 


$^; Exceptions: 


v 
${#*} and ${#@} give the number of positional parameters. 


9 For an array, ${#array[*]} and ${#array[@ ]) give the number of elements in 
the array. 


Example 10-9. Length of a variable 


1 #!/bin/bash 
2 length.sh 
3 
4 E NO ARGS-65 
5 
6 if [ $# -eq 0 ] # Must have command-line args to demo script. 
7 then 
8 echo "Please invoke this script with one or more command-line arguments." 
9 exit $E NO ARGS 
TOREA 
L 
12 var0l=abcdEFGH28ij 
13 echo "var01 = ${var01}" 
14 echo "Length of var01 = S${#var01}" 
15 # Now, let's try embedding a space. 
16 var02-"abcd EFGH281ij" 
17 echo "var02 = S$(var02)" 
18 echo "Length of var02 = S${#var02}" 
19 
20 echo "Number of command-line arguments passed to script = ${#@0}" 
21 echo "Number of command-line arguments passed to script = S${#*}" 
27 
23 exit 0 


${var#Pattern}, ${var##Pattern} 
${var#Pattern} Remove from $var the shortest part of $Pattern that matches the front end 
of Svar. 
${var##Pattern} Remove from $var the longest part of $SPattern that matches the front end 


of Svar. 


A usage illustration from Example A-7: 


1 # Function from "days-between.sh" exampl 

2 # Strips leading zero(s) from argument passed. 

3 

4 strip leading zero () 4$ Strip possible leading zero(s) 

D 1 #+ from argument passed. 

6 return=$ {1#0} u mie US rerecg jee) YSU == pasee auc). 

T # The "O" is what to remove from "$1" -- strips zeros. 


Manfred Schwarb's more elaborate variation of the above: 


1 strip leading zero2 () 4 Strip possible leading zero(s), since otherwise 

ZEE # Bash will interpret such numbers as octal values. 

3 shopt -s extglob # Turn on extended globbing. 

4 local val=${1l##+(0)} # Use local variable, longest matching series of 0's. 
3 shopt -u extglob # Turn off extended globbing. 

6 .Strip leading zero2-$(val:-0) 

7 # If input was 0, return O instead of "". 


om) 
Another usage illustration: 


1 echo 'basename S$PWD' Basename of current working directory. 
2 echo "S{PWD##*/}" Basename of current working directory. 
3 echo 

4 echo "basename $0" Name of script. 

5 Gelao SO) Name of script. 

6 echo "S{O##*/}" Name of script. 

7 echo 

8 filename-test.data 

9 echo "S{filename##*.}" data 

10 Extension of filename. 


${var%Pattern}, ${var%%Pattern} 
${var % Pattern} Remove from $var the shortest part of SPattern that matches the back end 
of Svar. 
${var % % Pattern} Remove from $var the longest part of $Pat tern that matches the back end 


of Svar. 


Version 2 of Bash added additional options. 


Example 10-10. Pattern matching in parameter substitution 


1 #!/bin/bash 
2 4 patt-matching.sh 
3 
4 # Pattern matching using the # ## $ $$ parameter substitution operators. 
3 
6 varl=abcd12345abc6789 
7 patternl-a*c # * (wild card) matches everything between a - c. 
8 
9 echo 
10 echo "varl = $varl" 4 abcd12345abc6789 
Ii Seno Wyer = Seul # abcd12345abc6789 
12 # (alternate form) 
13 echo "Number of characters in ${varl} = ${#varl}" 
14 echo 
15 
16 echo "patternl = $patternl" # a*c (everything between 'a' and 'c') 
JL cexeleo. n n 
18 echo 'S{varl#Spatternl} =! "S{varl#S$patternl1}" # d12345abc6789 
19 # Shortest possible match, strips out first 3 characters  abcd12345abc6789 
20 # ALICE |= 
21 echo 'S{varl##Spatternl} -' "S{varl##$patternl1}" # 6789 
22 # Longest possible match, strips out first 12 characters  abcd12345abc6789 
23 # ups | cc | 
24 
25 echo; echo; echo 
26 
27 pattern2-b*9 # everything between 'b' and '9' 
28 echo "weuclh = Sweueil # Still  abcd12345abc6789 
29 echo 
30 echo "pattern2 = Spattern2" 
Sil echo U " 
32 echo 'S{varl%pattern2} =' "S{varl%Spattern2}" # abcd12345a 
33 4 Shortest possible match, strips out last 6 characters  abcd12345abc6789 
34 # 1o j====] 
35 echo 'S{varl%%pattern2} -' "S{varl%%S$pattern2}" # a 
36 # Longest possible match, strips out last 12 characters  abcd12345abc6789 
37 # retra || m | 
38 


39 4 Remember, # and ## work from the left end (beginning) of string, 


40 # $ and $$ work from the right end. 
42 echo 


44 exit O 


Example 10-11. Renaming file extensions: 


!/bin/bash 
rfe.sh: Renaming file extensions. 


rfe old_extension new_extension 


Example: 
ho renema all = Gohe miles in vorkiwe Chihrectoci To "53 9, 
rfe gif jpg 


E BADARGS-65 


case $4 in 
0|1) # The vertical bar means "or" in this context. 
echo "Usage: 'basename $0' old file suffix new file suffix" 
exit SE BADARGS # If O0 or 1 arg, then bail out. 


Gt 


esac 


NPRPRPRP PPP PP 
SCOMIDTUHRWNHFOWCMIDGUBRWNE 


21 for filename im 95.91 

22 # Traverse list of files ending with 1st argument. 

23 do 

24 mv $filename ${filename%$1}$2 

25 # Strip off part of filename matching lst argument, 
26 #+ then append 2nd argument. 

27 done 

28 

2S) cw 0 


Variable expansion / Substring replacement 
These constructs have been adopted from ksh. 
${var:pos} 
Variable var expanded, starting from offset pos. 
${var:pos:len} 
Expansion to a max of len characters of variable var, from offset pos. See Example A-13 for an 
example of the creative use of this operator. 
${var/Pattern/Replacement } 
First match of Pattern, within var replaced with Replacement. 


If Replacement is omitted, then the first match of Pattern is replaced by nothing, that is, 
deleted. 

${var//Pattern/Replacement } 
Global replacement. All matches of Pattern, within var replaced with Replacement. 


As above, if Replacement is omitted, then all occurrences of Pattern are replaced by nothing, 
that is, deleted. 


Example 10-12. Using pattern matching to parse arbitrary strings 


#!/bin/bash 


varl=abcd-1234-defg 
echo "varl = $varl" 


t=S{varl#*—*} 

echo "varl (with everything, up to and including first - stripped out) = St" 
# t=S{varl#*-} works just the same, 

#+ since # matches the shortest string, 

#+ and * matches everything preceding, including an empty string. 

# (Thanks, Stephane Chazelas, for pointing this out.) 


t=S{varl##*—*} 
echo “Ike varl eontaims a Y=, returne Samoty tiii so varl = $t" 


SS vecon E] 
echo "varl (with everything from the last - on stripped out) = $t" 


Mop pop opp PPP PY 
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echo 


N N 
N E 


# 
path, name-/home/bozo/ideas/thoughts.for.today 
# 
echo "path_name = Spath_name" 
t=${path_name##/*/} 
echo "path name, stripped of prefixes = $t" 

Same effect as t-'basename $path name' in this particular case. 

t-$(path name$/); t=S{t##*/} is a more general solution, 
+ but still fails sometimes. 

If $path name ends with a newline, then 'basename $path name' will not work, 
t but the above expression will. 

(Thank Sy (Sis) 


N 
[99 


NNN PN 
=] fe» (On gem 


N 
[99] 


C9 CO NO 
[— «y Wo) 
f 


w 
N 
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t=${path_name%/*.*} 

Same effect as t= dirname $path name' 

echo "path name, stripped of suffixes = $t" 

Hüte akil reil aia Some Cases, sue as V3 rooh se VrO M. M/s 
Removing suffixes, especially when the basename has no suffix, 

+ but the dirname does, also complicates matters. 

hank Sy Soles) 


C9 CO CO CO CO CO CO 
«o 0 -—-1 OY O1 4 CO 


[i 
e 


[i 
fe 


ecno 


$ {path_name:11} 

echo "Spath_name, with first 11 chars stripped off = $t" 

t-$(path name:11:5) 

echo YSparn meme; wita dessen LI chars mWExcaljegeie el (rir, diesem 5 = Sie! 


‘D Aa Ah Ss iS fes gs os 
Wo (99) al (ox Gi ge. wy [S5 
ct 
ll 


no 


U O1 
ERO: 
oO 
Q 


O1 

N 

ct 
Il 


${path_name/bozo/clown} 

echo "Spath_name with \"bozo\" replaced by \"clown\" = $t" 
t=${path_name/today/} 

echo "Spath_name with \"today\" deleted = $t" 

$ {path_name//o/0} 

echo "Spath_name with all o's capitalized = $t" 
t=${path_name//o/} 

echo "Spath_name with all o's deleted = $t" 


(Onr Gab (nh On Gn toni (On 
«oO O0 —1 OY oO iS CO 
ct 
Il 


Orv OV 
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exit 0 


${var/#Pattern/Replacement } 


If prefix of var matches Pattern, then substitute Replacement for Pattern. 
${var/%Pattern/Replacement } 
If suffix of var matches Pattern, then substitute Replacement for Pattern. 


Example 10-13. Matching patterns at prefix or suffix of string 


#!/bin/bash 
# var-match.sh: 
# Demo of pattern replacement at prefix / suffix of string. 


v0-abcli234zip1234abc # Original variable. 
echo "v0 = $v0" # abcl234zip1234abc 
echo 


# Match at prefix (beginning) of string. 
vl-$(v0/tabc/ABCDEF) abci234zipl1234abc 
Isl 

exelao. yil = Syl? ABCDEF1234zip1234abc 
E 


# Match at suffix (end) of string. 


Mop p opp PPP PP 
O (o 0» -1 O Oi i (). NM HP O xo 0 -1 O O1 4 CQ Io E 


v2=$ {v0/%abc/ABCDEF } abc1234zipl23abc 
=| 
echo "v2 = S$v2" abc1234zip1234ABCDEF 
===>] 
21 echo 
22 
Aon kj 
24 # Must match at beginning / end of string, 
25 #+ otherwise no replacement results. 
26 # 
27 v3=S{v0/#123/000} # Matches, but not at beginning. 
28 echo "v3 = $v3" # abcl234zip1l234abc 
Z9 # NO REPLACEMENT. 
30 v4=S${v0/%123/000} # Matches, but not at end. 
31 echo "v4 = $v4" # abcl234zipl1234abc 
32 # NO REPLACEMEN 
33 
34 exit 0 


${!varprefix*},${!varprefix@} 
Matches names of all previously declared variables beginning with varprefix. 


dL Giz "rente: aber sel varian oN (olin) slicvelakietevele, setSuEehersiaversy, Jolbhe Wurm. ~l 7 fone (Ch, 
2 # Bash, version 2.04, adds this feature. 

3 

4 xyz23=whatever 

5 xyz24= 

6 

7 a=S{!xyz*} # Expands to *names* of declared variables 
OU EA ^ + beginning with "xyz". 

Se echo ae Sa # a = xyz23 xyz24 
10 a=S${!xyz@} # Same as above. 
Li ximo Ver = Se" # a = xyz23 xyz24 
12 
1S) exeleigy Moses 
14 
15 abc23-something els 
16 b=S{!abc*} 
ly echo Wis = Slo # b = abc23 
18 c=${!b} # Now, the more familiar type of indirect reference. 


19 echo $c # something else 


Notes 


[1] IfS$parameter is null in a non-interactive script, it will terminate with a 127 exit status (the Bash error 
code for "command not found"). 
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Chapter 11. Loops and Branches 


What needs this iteration, woman? 
--Shakespeare, Othello 


Operations on code blocks are the key to structured and organized shell scripts. Looping and branching 
constructs provide the tools for accomplishing this. 


11.1. Loops 


A loop is a block of code that iterates [1] a list of commands as long as the loop control condition is true. 


for loops 


for argin [list] 
This is the basic looping construct. It differs significantly from its C counterpart. 


for argin[list] 


do 


command (s)... 


done 


$^; During each pass through the loop, arg takes on the value of each successive variable 


in the list. 

i for are aim “"Swairl! YSware2”? Vwi oc, UkyeueiNU 

2 4 In pass 1 of the loop, arg = Svarl 

3 # In pass 2 of the loop, arg = $var2 

4 # In pass 3 of the loop, arg = $var3 

Sets eres 

6 # In pass N of the loop, arg = $varN 

7 

8 # Arguments in [list] quoted to prevent possible word splitting. 


The argument 1 ist may contain wild cards. 


If do is on same line as for, there needs to be a semicolon after list. 


for argin[1ist];do 


Example 11-1. Simple for loops 


PRPRPPPP PEP YE 
XO O0 -1 OY O1 i$ (0. I9. IS. O «o 0 -1 OY O1 iS CQ) IO ES 


N N 
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#!/bin/bash 
# Listing the planets. 


for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto 
do 

echo $planet # Each planet on a separate line. 
done 


echo; echo 


for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto" 
# All planets on same line. 
# Entire 'list' enclosed in quotes creates a single variable. 
# Why? Whitespace incorporated into the variable. 


do 
echo $planet 
done 


echo; echo "Whoops! Pluto is no longer a planet!" 


exec O 


Each [list] element may contain multiple parameters. This is useful when processing parameters 
in groups. In such cases, use the set command (see Example 15-16) to force parsing of each [1ist] 


element and assignment of each component to the positional parameters. 


Example 11-2. for loop with two parameters in each [list] element 


#!/bin/bash 
# Planets revisited. 


# Associate the name of each planet with its distance from the sun. 


it 
2 
3 
4 
5 
6 for planet in "Mercury 36" "Venus 67" "Earth 93" "Mars 142" 
7 do 
8 set -—- $planet #  Parses variable "planet" 
9 #+ and sets positional parameters. 
10 The "--" prevents nasty surprises if $planet is null or 
1 + begins with a dash. 

2 

3 

4 

5 

6 

7 

8 


May need to save original positional parameters, 

+ since they get overwritten. 

One way of doing this is to use an array, 
original, params- ("$Q") 


echo "$1 82,000,000 miles trem tae sum” 
LG See two tabs concatenate zeroes onto parameter $2 


AD a. (Hinks, SoCo, wor acleliieslomel lleiesir eae sem c.) 


24 exit 0 


"Jupiter 483" 


A variable may supply the [list] ina for loop. 


Example 11-3. Fileinfo: operating on a file list contained in a variable 


#!/bin/bash 
# fileinfo.sh 


FILES="/usr/sbin/accept 

/usr/sbin/pwck 

/usr/sbin/chroot 

/usr/bin/fakefile 

/ Sbin/badblocks 

/ Sbin/ypbind" # List of files you are curious about. 


echo 


fos ipie aia Su 


El 
n 


Mop p op op op p ppt 
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do 
ax (| 2 =e Saule" ] # Check if file exists. 
then 
echo WSiriile Coes NOE Exist. UP Seino 
continue # On to next. 
2I iE 3. 
22 
23 ig =i Sitiie || awk "4 prime Se Y Plerotes UP iG jeu 
24 whatis 'basename $file # File info. 


# Threw in a dummy file, /usr/bin/fakefile. 


# Print 2 fields. 


25 # Note that the whatis database needs to have been set up for this to work. 
26 13 IC) Clo tars, as root rin Sie Morin /alsiylmene iss 

2 echo 

28 done 

29 

30 ces () 


If the [list] in a for loop contains wild cards (* and ?) used in filename expansion, then globbing 
takes place. 


Example 11-4. Operating on files with a for loop 


1 #!/bin/bash 
2 1» iisit-glolo.sing Cemeretime [iise] im a ror=-locp;, teine "glslsau" 
3 
4 echo 
5 
G 3Egue seal, alin = 
7 # ^ Bash performs filename expansion 
8 #+ on expressions that globbing recognizes. 
9 clo 
10 ig  —3 USiile"U Ge igisics all riles coe SPW (@ulemeine qlbxexeo y) - 
al # Recall that the wild card character "*" matches every filename, 
12 #+ however, in "globbing," it doesn't match dot-files. 
43 
14 # If the pattern matches no file, it is expanded to itself. 
LS # To prevent this, set the nullglob option 
Ls #+ (shopt = mule). 
L7 s; linguaks, SiC. 
18 done 
19 
20 echo; echo 
eal 
22 HOw sile aim |[qee]| * 
23 CS 
24 iam Sie Silke # Removes only files beginning with "j" or "x" in SPWD. 
25 Cho “Remowacl ile Wy US ey". 
26 done 
27 
28 echo 
29 
30 exit 0 


Omitting the in [list] part of a for loop causes the loop to operate on $@ -- the positional 
parameters. A particularly clever illustration of this is Example A-15. See also Example 15-17. 


Example 11-5. Missing in [list] in a for loop 


echo -n "Sa " 
done 


1 #!/bin/bash 

2 

3 # Invoke this script both with and without arguments, 
4 #+ and see what happens. 

5 

© BOR a 

7 do 

8 

E 


LO 

11 # The Yim list' missing, therefore the loop operates on '$Q' 
12 #+ (command-line argument list, including whitespace). 

13 

14 echo 

15 

16 exit O 


It is possible to use command substitution to generate the [list] in a for loop. See also Example 
16-54, Example 11-10 and Example 16-48. 


Example 11-6. Generating the [list] in a for loop with command substitution 


1 #!/bin/bash 

2 4 toe-locociwiel. Sins iom-logp waitin [List] 
3 #+ generated by command substitution. 

4 

>) NUMBERS UO Sco d aos 

6 

J zor iolbinlovsie im echo SNUMSERS  ; cor inybimloyse im 9 7 3 E 97-53 
8 do 

9 echo -n "$number " 
10 done 
JT 
12 echo 
13 exit 0 


Here is a somewhat more complex example of using command substitution to create the [1ist]. 


Example 11-7. A grep replacement for binary files 


1 #!/bin/bash 

2 bin-grep.sh: Locates matching strings in a binary file. 
3 

4 A "grep" replacement for binary files. 

5 Similar effect to "grep ag 

6 

7 E BADARGS-65 

8 E NOFILE-66 

E 

Jig sc [| $$ ne 2 | 

11 then 

2 echo "Usage: 'basename $0° search string filename" 
1.3 exit $E BADARGS 

Jd su 

15 

16 sx | 4 =i VS2w || 

17 then 

18 echo "File \"S2\" does not exist." 

L exit $E NOFILE 
2(0 sta 
ait 
22 
29 lj" WI # Per suggestion of Anton Filippov. 
24 # was: IFS="\n" 
25 toe wore! am SI seringe "S2" | Gree "Si" ) 


26 # The "strings" command lists strings in binary files. 
27 # Output then piped to "grep", which tests for desired string. 
28 do 


29 echo $word 


30 done 

Sil 

32 # As S.C. points out, lines 23 - 30 could be replaced with the simpler 
Sst ous swane US. | eres YSU" | iue =a YSIS? pages] V 

34 

35 

36 # Try something like "./bin-grep.sh mem /bin/ls" 

37 #+ to exercise this script. 

38 

39 exit 0 


More of the same. 


Example 11-8. Listing all users on the system 


#!/bin/bash 
# userlist.sh 


PASSWORD_FILE=/etc/passwd 
ipei # User number 


for name in $(awk 'BEGIN{FS=":"}{print $1)' < "SPASSWORD_FILE" ) 
# Field separator = BE SNE 
Ernte GSE Erelid: SAONA 
# Get input from password file 
do 

echo "USER #$n = $name" 

let m a AN 
done 


AKKRKKRKAKAKRAKRAAAARAA 


"d 
= 


= root 
bin 
daemon 


(oor | copy Gab GSS (es) [Sy [— (ee We) es) cp (ex (Gap gem (63) [Sp [= 


q 
n 

(a MB a ARa 

J Ww 

de de He 
N 
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n 
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USER #30 = bozo 


27 4 How is it that an ordinary user (or a script run by same) 
28 #+ can read /etc/passwd? 
29 # Isn't this a security hole? Why or why not? 


Yet another example of the [1ist] resulting from command substitution. 


Example 11-9. Checking all the binaries in a directory for authorship 


1 #!/bin/bash 

2 4 findstring.sh: 

3 4 Find a particular string in the binaries in a specified directory. 

4 

5 directory-/usr/bin/ 

6 fstring-"Free Software Foundation" # See which files come from the FSF. 
7 

9) icone cile ta S( riae Sclimeciroeny -tyo i -nane V" || meum ) 

9 do 

10 Strings -r Stile | grep "Sfstrzng" | sed -e "StSdirectoryss" 


© (e) (er c wy Gal de» Ge [ES p 


Nh d 


N N 
N EB 


# In the "sed" expression, 
#+ it is necessary to substitute for the normal "/" delimiter 
#+ because "/" happens to be one of the characters filtered out. 


# Failure to do so gives an error message. (Try it.) 
done 
exit $? 


Exercise (easy): 


Convert this script to take command-line parameters 
+ kor Schimecicomy aimcl Sirsicienine). 


A final example of [List] / command substitution, but this time the "command" is a function. 


generate_list () 
{ 
echo "one two three" 
} 
for word in $(generate_list) # Let "word" grab output of function. 
do 
echo "Sword" 
done 
# one 
# two 
# three 


The output of a for loop may be piped to a command or commands. 


Example 11-10. Listing the symbolic links in a directory 


#!/bin/bash 
# symlinks.sh: Lists symbolic links in a directory. 


directory- sd pwa 


Defaults to current working directory, 
+ if not otherwise specified. 
Equivalent to code block below. 
ARGS=1 # Expect one command-line argument. 
if [ $4 -ne "SARGS" ] 5h Joie doute. db euse 6 
then 
directory= pwd' # current working directory 
else 
directory-$1 
ftu 


echo "symbolic links in directory \"Sdirectory\"" 


ioe dem he: ain) WS rine Schireccor -cyose I jy # -type 1 = symbolic links 
do 
seine Wei 
done | sort # Otherwise file list is unsorted. 
# Strictly speaking, a loop isn't really necessary here, 


#+ since the output of the "find" command is expanded into a single word. 
# However, it's easy to understand and illustrative this way. 


30 
Sal 
32 
33 
34 
39 
36 
37 
38 
3$ 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sil 
52 
52 
54 
55 
56 
5 
58 
59 
60 
61 
62 
63 
64 


# As Dominik 'Aeneas' Schnitzer points out, 

#+ failing to quote $( find Sdirectory -type L ) 
#+ will choke on filenames with embedded whitespac 
# containing whitespace. 


exe O 


# 
# Jean Helou proposes the following alternative: 


echo Yeyiloolic lis sim cirectory \Sclineccony\"" 

# Backup of the current IFS. One can never be too cautious. 
OLDIFS=SIFS 

IS 


tole rile lm S(sedinel Scliiecieoiay -typa L -priori VY SjoSmi Ss”) 
do # RARKRAKAKRKRAKAKRAARAARKRANA 
echo "Sfile" 
done|sort 


# And, James "Mike" Conley suggests modifying Helou's code thusly: 


OLDIFS-SIFS 
IFS='' # Null IFS means no word breaks 
iG file aim GS fime Sdirectory -type l ) 
do 

echo $file 

done | sort 


# This works in the "pathological" case of a directory name having 


#+ an embedded colon. 
# "This also fixes the pathological case of the directory name having 
#+ a colon (or space in earlier example) as well." 


The stdout of a loop may be redirected to a file, as this slight modification to the previous example 
shows. 


Example 11-11. Symbolic links in a directory, saved to a file 


O er (oo. — (ex; Gy des ies IS) [5 €» (Wer Cs cL ex) (Gn PSS (93. [eS [5 


NO +t 


#!/bin/bash 
# symlinks.sh: Lists symbolic links in a directory. 


OUTFILE-symlinks.list # save file 


directory-$(1-'pwd') 
# Defaults to current working directory, 
#+ if not otherwise specified. 


eche "guloolic links alin directory \YSdiresctory YY > "SQUID LUHY 


Gho W U Sa US(OQUEDS3USU 
tow tile iin UG same! Scllwectormy tyes L )" -type 1 = symbolic links 
do 

echo "Sfile" 
done | sort >> "SOUTFILE" stdout of loop 
# MG D DE redirected to save file. 
exit 0 


There is an alternative syntax to a for loop that will look very familiar to C programmers. This 
requires double parentheses. 


Example 11-12. A C-style for loop 


I | 


PRP PE + m oH 
O00 -1 OY Ui 4 CQ). 9. IB. O (0 00 -1 O Or dS (QN ES 


COPD ssp des dese SNS dv esr dur ND W i 
H &) Wey (Ge! sal ten al dex CO RS) p^ €» We 


WWWWW CO CO CO 
Gey (or sa) cx Oe de Ge) I» 


op 
i= €» 


CO Or; Pb ee PS a us 
[= (x We) (e sa] rex Gal de s [5S 


U1 
N 
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#!/bin/bash 


# Multiple ways to count up to 10. 


echo 


# Standard synta 


for m im d 2 3 4 56 7 $9 9 1 


do 
echo -n 
done 


usi " 


echo; echo 


X. 


# + 


# Using "seq" 
for a in 'seq 10 
do 

echo -n 
done 


mes " 


echo; echo 


# + 


# Using brace ex 
# Bash, version 
one mk st {iL 210] 
do 

echo -n 
done 


mo " 


echo; echo 


pansion 
Soe 


# + 


s New, lew"s co 


LIMIT=10 


for 
do 

echo -n 
done 


vsa " 


echo; echo 


the same, 


LIMIT ; att+)) 


# Double parentheses, 


# A construct borrowed from 


dr 


using C-like syntax. 


and 


"LIMIT" with no 


VIRGINS V 


Mim 


# + 


# Let's use the 


do # The comma chains together operations. 


for ((a=1, b=1; 
echo n "$a-$b 

done 

echo; echo 

exit 0 


+ 


C "comma operator" to increment two variables simultaneously. 


a <= LIMIT ; 


act, 


b++) ) 


while 


See also Example 27-16, Example 27-17, and Example A-6. 


Now, a for loop used in a "real-life" context. 


Example 11-13. Using efax in batch mode 


! /bin/bash 
Faxing (must have 'efax' package installed). 


EXPECTED ARGS-2 

E BADARGS-85 

ODEM_PORT="/dev/ttyS2" # May be different on your machine. 
EOI PCMCIA modem card default port. 


if [ $# -ne SEXPECTED_ARGS ] 

Check for proper number of command-line args. 
then 
echo 
exit 


e 


Usage: ~basename $0? phones text-file" 
E BADARGS 


Xr 


ital 


PPP Pt H 
O0 -1 OY O1 4 C). IN. EB. O (0 O0 -1 O O1 iS (Q I9 BS 


ix [ 2 es wS2" ] 

then 
exo mule $2 se joi e is ule. 
# File is not a regular file, or does not exist. 
exit $E BADARGS 

it 


NO NO NO PS F 
eS [i [9 c9 el 


NO ON 
ow 


fax make $2 # Create fax-formatted files from text files. 


N N 
-] Oy 


Be BNS aim Axes $2 595) #  Concatenate the converted files. 
# Uses wild card (filename "globbing") 
#+ in variable list. 


N 
© 


do 
imb uerb S Sfilen 
done 


w CO hO 
[— Sy We 


Ww 
N 


efax —d "SMODEM PORT"  -t "TSI" Sfill 4 Finally, do the work. 
# Trying adding -ol if above line fails. 


# As S.C. points out, the for-loop can be eliminated with 
# efax -d /dev/ttyS2 -ol -t "T$1" $2.0* 
#+ but it's not quite as instructive [grin]. 


CO0 CO CO CO CO CO CO 
«oO 0 ITA BW 


PD 
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exit $? # Also, efax sends diagnostic messages to stdout. 


This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is 
true (returns a 0 exit status). In contrast to a for loop, a while loop finds use in situations where the 
number of loop repetitions is not known beforehand. 


while [ condition] 
do 
command (s)... 


done 


The bracket construct in a while loop is nothing more than our old friend, the test brackets used in an 
if/then test. In fact, a while loop can legally use the more versatile double-brackets construct (while [[ 


condition ]]). 


As is the case with for loops, placing the do on the same line as the condition test requires a 


semicolon. 


while[ condition];do 


Note that the test brackets are not mandatory in a while loop. See, for example, the getopts construct. 


Example 11-14. Simple while loop 


-n suppresses newline. 


=p (wee ar 12) 


, to separate printed out numbers. 
also works. 
also works. 
also works. 


Various other methods also work. 


1 #!/bin/bash 
2 
3 var0-20 
4 LIMIT-10 
5 
6 walle | Yeyarou -ike WEIMA i] 
7 A ^ 
8 Spaces, because these are "test-brackets" 
9) telo) 
AL) echo -m "Svar0 " # 
L1 + es Spac 
12 
13 var0= expr svaro + 1° # var0=S (($var0+1) ) 
14 # varo 
15 # let "varO0 += 1 
16 done # 
17 
18 echo 
19 
AQ) exite © 


Example 11-15. Another while loop 


1 #!/bin/bash 

2 

3 echo 

4 # Equivalent to: 

5 while [ "$varl" != "end" ] # while test "$varl" != "end" 
6 do 

7 echo "Input variable #1 (end to exit) " 

8 read varl # Not 'read $varl' (why?). 

9 echo "variable #1 = $varl" # Need quotes because of "#" 
3L) # If input is 'end', echoes it her 
L1 # Does not test for termination condition until top of loop. 
1,2 echo 
13 done 
14 
15 exit 0 


A while loop may have multiple conditions. Only the final condition determines when the loop 
terminates. This necessitates a slightly different loop syntax, however. 


Example 11-16. while loop with multiple conditions 


#!/bin/bash 


varl=unset 
previous-$varl 


while echo "previous-variable = $previous" 
echo 
previous-$varl 
[ "Svarl" != end ] 4 Keeps track of what Svarl was previously. 


# Four conditions on "while", but only last one controls loop. 
# The *last* exit status is the one that counts. 
do 
echo "Input variable #1 (end to exit) " 
read varl 
echo “variable l = gyari” 
done 


# Try to figure out how this all works. 
# It's a wee bit tricky. 


[= «xy Wey si | xexy Gal dex Tes) [e$ [- Gy) (e) (sr c3] GR (ab dE (es) [e5h J 


N N + 


exit 0 


As with a for loop, a while loop may employ C-style syntax by using the double-parentheses construct 
(see also Example 8-5). 


Example 11-17. C-style syntax in a while loop 


f!/bin/bash 
# wh-loopc.sh: Count to 10 in a "while" loop. 


LIMIT-10 
a-1 


wale [ “Sa! -le Sui | 
do 
Cho —m "Sg V 
let "a+=1" 
done # No surprises, so far. 


echo; echo 


# + + 


# Now, repeat with C-like syntax. 


NPRPRPP RPP PP PY 
COMIDTHRWNHHOWCAIDUBRWNE 


(f(a = 1)) # a-1 
# Double parentheses permit space when setting a variable, as in C. 
2 
22 while (( a <= LIMIT )) # Double parentheses, and no "S" preceding variables. 
23. ClO) 
24 echo -n "$a " 
25 ((a += 1)) # let "a+t=1" 


26 # Yes, indeed. 
27 # Double parentheses permit incrementing a variable with C-like syntax. 


28 done 

29 

30 echo 

ES 

32 4 C programmers can feel right at home in Bash. 
33 

34 exit 0 


Inside its test brackets, a while loop can call a function. 


a 
ll 
e 


Gomelsicskein O) 


{ 
( (ier) ) 


aie [| $E -1r mS] 


then 

return 0 # true 
else 

return 1 # false 
£3 


c 


while condition 
# KRAKAKRAAAA 
# PAC ELGM call == rour loca 3xiEGUE GEIL s 
do 
exiguo WEIL goings E = Su" 
done 


NPRPRPPRP PPP PY 
O (o 00 -1 O Oi i$ (). M HIS. O xo 0 -1 O O1 iS CQ I E 


NO ON 
[R3 [= 


Sigil gorneg 
SiELILIL quoe 
Siew gorneg 
Siew (Goes 


N 
[99 


N 
T 

(nr (er im (er 
ll 

duy w IS) i 


N 
[91] 


Similar to the if-test construct, a while loop can omit the test brackets. 


while condition 
do 

command(s) 
done 


dex 9 Jm» I 


By coupling the power of the read command with a while loop, we get the handy while read construct, 
useful for reading and parsing files. 


cat $filename | # Supply input from a file. 
while read line # As long as there is another line to read 
do 


while read valu # Read one data point at a time. 
do 

rt-$(echo "scale-$SC; Srt + $value" | bc) 

(( ctt+ )) 
done 


[s 
© (e) (Ger =) rex; Gal dex Ge [eS p 


[Es 
=A 


PRR 
Dm WN 


until 


15 am-$(echo "scale-$SC; $rt / $ct" | bc) 


16 

17 echo seny Tecur s # This function "returns" TWO values! 
18 Ww  Qewutuious imus little trick wall mot work i Set > 255! 

ig # To handle a larger number of data points, 

20 #+ simply comment out the "return Sct" above. 

Zi p «V Slenceu st ke # Feed in data file. 


8^) A while loop may have its st din redirected to a file by a < at its end. 


A while loop may have its st din supplied by a pipe. 


This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is 
false (opposite of while loop). 


until [| condition-is-true] 
do 

command (s)... 
done 


Note that an until loop tests for the terminating condition at the top of the loop, differing from a 
similar construct in some programming languages. 


As is the case with for loops, placing the do on the same line as the condition test requires a 
semicolon. 


until [| condition-is-true];do 


Example 11-18. until loop 


1 #!/bin/bash 

2 

3 END_CONDITION=end 

4 

5 until [| "Svarli" = "SEND CONDITION" ] 

6 Tests condition here, at top of loop. 

T co 

8 echo "Input variable #1 " 

9 echo "(SEND_CONDITION to exit)" 

10 read varl 

La acho “warialole pl = Swarr” 

L2 echo 

13 done 

14 

15 # # 
16 

L7 d AS wilca  "Uitoae " incl warts lowes, 

le spe am Vum l“ loge joereuits ColikS teet (COSI DIES. 
19 
20 LIMIT=10 
21 var=0 
22 
29. wnel ((( were 2 METIE D») 
24 do # ^^ n^ i e No brackets, no $ prefixing variables. 
25 echo -n "Svar " 
26 (Coraes J) 
27 done w OL 2AAS GT sg 9 10 
28 


N 
Ko) 


30 es Q 


How to choose between a for loop or a while loop or until loop? In C, you would typically use a for loop 
when the number of loop iterations is known beforehand. With Bash, however, the situation is fuzzier. The 
Bash for loop is more loosely structured and more flexible than its equivalent in other languages. Therefore, 
feel free to use whatever type of loop gets the job done in the simplest way. 


Notes 


[1] Iteration: Repeated execution of a command or group of commands, usually -- but not always, while a 
given condition holds, or until a given condition is met. 
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11.2. Nested Loops 


A nested loop is a loop within a loop, an inner loop within the body of an outer one. How this works is that 
the first pass of the outer loop triggers the inner loop, which executes to completion. Then the second pass of 
the outer loop triggers the inner loop again. This repeats until the outer loop finishes. Of course, a break 
within either the inner or outer loop would interrupt this process. 


Example 11-19. Nested Loop 


1 #!/bin/bash 

2 4 nested-loop.sh: Nested "for" loops. 

3 

4 outer-1 # Set outer loop counter. 
5 

6 # Beginning of outer loop. 

7 ie &m im d 2 9 4 5 


8 do 

9 echo "Pass Souter in outer loop." 

10 pim " M 

dat inner=1 # Reset inner loop counter. 

12 

13 # 

14 # Beginning of inner loop. 

15 tor b im l 2 34 5 

16 do 

17) echo "Pass Sinner in inner loop." 

its} let "innert+t=1" # Increment inner loop counter. 
JL) done 

20 # End of inner loop. 

2 # 

22 

25 let "outer+=1" # Increment outer loop counter. 
24 echo # Space between output blocks in pass of outer loop. 
25 done 

26 4 End of outer loop. 

Z7 

28 exit 0 


See Example 27-11 for an illustration of nested while loops, and Example 27-13 to see a while loop nested 
inside an until loop. 
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11.3. Loop Control 


Tournez cent tours, tournez mille tours, 
Tournez souvent et tournez toujours . . . 


--Verlaine, "Chevaux de bois" 


Commands affecting loop behavior 


break, continue 
The break and continue loop control commands [1] correspond exactly to their counterparts in other 
programming languages. The break command terminates the loop (breaks out of it), while continue 
causes a jump to the next iteration of the loop, skipping all the remaining commands in that particular 
loop cycle. 


Example 11-20. Effects of break and continue in a loop 


RS [S3 [RS [RS [ES qp qp og eee Irt 
W [nox f c9» (Wer (Ger a] (exp (wb dE (ESL [sp [— SS) (Wen (9 c (ex; (nl dE (9. fe» [3 


NO NO NO NY 
xi fey) (wp GS 


N 
[99] 


C9 CO CO CO CO CO CO DN 
fex (Ow dE (eS) [e [3 €0» uel 


SH 


Ww 
[99] 


39 
40 
41 
42 
43 


#!/bin/bash 
LIMIT-19 # Upper limit 


echo 
echo WiEieiimeame Numosrs L throwch 20 (lowe imo 3 ane) 11b) 


a=0 


while [ $a -le "SLIMIT" ] 


do 

a-$(($a*1)) 

ad | "Sa" ee 3 J| || [ "Se? -eeg di | i mxeiluces 3 aime iil. 
then 

continue # Skip rest of this particular loop iteration. 

ica 

cecho =a "Sa U # This will not execute for 3 and 11. 
done 


# Exercise: 
# Why does the loop print up to 20? 


echo; echo 
echo Printing Numbers 1 through 20, but something happens after 2. 
HGH EERE EE HHH HE EE EE EHH HE EE EEE EE HE EE EE EE ERE HE EER EE EE HE EEE 


# Same loop, but substituting 'break' for ‘continue’. 


a=0 
vaile f “Sav =le VS T 

do 

a=$ (($a*1)) 

die [p VS." ge 2 ] 

then 

break # Skip entire rest of loop. 

ie aL 


44 acing —m "Sg V" 
45 done 

46 

47 echo; echo; echo 
48 

AS esgic (0) 


The break command may optionally take a parameter. A plain break terminates only the innermost 
loop in which it is embedded, but a break N breaks out of N levels of loop. 


Example 11-21. Breaking out of multiple loop levels 


1 #!/bin/bash 
2 4 break-levels.sh: Breaking out of loops. 


3 

4 # "break N" breaks out of N level loops. 

5 

G for ouirerlecs wa Jd 2 3 4 

7 do 

8 echo -n "Group Souterloop: Ww 

9 

10 # = Xr P iari UE 
dil For sbgusveselloyoyer alin) JL 2 3S} 4t 

12 do 

1 echo -n "$innerloop " 

14 

15 aie [p WUguwxesslloxp" -eg 3 | 

16 then 

Ey break # Try break 2 to see what happens. 
18 4 ("Breaks" out of both inner and outer loops.) 
9 1531 
20 done 
BAL # 
22 
225) echo 
24 done 
25 
26 echo 
27 
28 exit 0 


The continue command, similar to break, optionally takes a parameter. A plain continue cuts short 
the current iteration within its loop and begins the next. A continue N terminates all remaining 
iterations at its loop level and continues with the next iteration at the loop, N levels above. 


Example 11-22. Continuing at a higher loop level 


1 #!/bin/bash 
2 # The "continue N" command, continuing at the Nth level loop. 


3 

4 fOr CUES 3um Jo QUI. LL iw W # outer loop 

5 do 

6 echo; echo -n "Group Souter: " 

7 

8 # LE = XS ES 
9 foie immer aim dL 2 $9. 4 5 6 7 9 9 IQ s auee logi 

10 do 

d.i 


2 sse [D] wesine ae 7 Sus Spuren" = "xx: qT 

3 then 

14 continue 2 # Continue at loop on 2nd level, that is "outer loop". 
15 # Replace above line with a simple "continue" 
6 # to see normal loop behavior. 

7 agal 

18 

Lg Seino =i WSatininere UL — st 7 49 S. ILO) Galil aoe cxelewo win Aene IIE 
20 done 
2 # 
22 
23 done 
24 
25 echo; echo 
26 
27 4 Exercise: 


28 4 Come up with a meaningful use for "continue N" in a script. 
29, 
30 exit 0 


Example 11-23. Using continue N in an actual task 


Albert Reiner gives an example of how to use "continue N": 


Suppose I have a large number of jobs that need to be run, with 
+ any data that is to be treated in files of a given name pattern in a 
+ directory. There are several machines that access this directory, and 
* I want to distribute the work over these different boxen. Then I 
* usually nohup something like the following on every box: 


Mop pop opp PPP PY 
O (o 00 -1 O Oi i (). NM P O (o 0 -1 O O1 4 CQ I9 E 


while true 
do 
itus. doi Shia s SLOG 
do 
E Ysa = Voase. cece!” || 1 Get 
beta=S{n#.iso. } 
[ -r .Iso.$beta ] && continue 
[ -r .lock.$beta ] && sleep 10 && continue 
lockfile -r0 .lock.$beta || continue 
echo -n "Sbeta: " “date” 
run-isotherm $beta 
23r date 
22 ls -alF .Tso.S$beta 
2.9 [ -r .Iso.$beta ] && rm -f .lock.$beta 
24 continue 2 
DIE) done 
26 break 
27 done 
28 


29 # The details, in particular the sleep N, are particular to my 
30 #+ application, but the general pattern is: 


El! 

32 while true 

33 ele 

34 for job in (pattern) 

35 do 

36 (job already done or running) && continue 

Sri (mark job as running, do job, mark job as done) 
38 continue 2 

B9 done 


40 break # Or something like `sleep 600' to avoid termination. 


41 done 


42 

43 This way the script will stop only when there are no more jobs to do 
44 #+ (including jobs that were added during runtime). Through the use 

45 #+ of appropriate lockfiles it can be run on several machines 

46 #+ concurrently without duplication of calculations [which run a couple 
47 #+ of hours in my case, so I really want to avoid this]. Also, as search 
48 #+ always starts again from the beginning, one can encode priorities in 
49 #+ the file names. Of course, one could also do this without "continue 2', 
50 #+ but then one would have to actually check whether or not some job 

51 #+ was done (so that we should immediately look for the next job) or not 
52 #+ (in which case we terminate or sleep for a long time before checking 
53 #+ for a new job). 


The continue N construct is difficult to understand and tricky to use in any 
meaningful context. It is probably best avoided. 


Notes 


[1] These are shell builtins, whereas other loop commands, such as while and case, are keywords. 


Prev Home Next 
Nested Loops Up Testing and Branching 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Chapter 11. Loops and Branches Nex 


— 


11.4. Testing and Branching 


The case and select constructs are technically not loops, since they do not iterate the execution of a code 
block. Like loops, however, they direct program flow according to conditions at the top or bottom of the 


block. 
Controlling program flow in a code block 


case (in) / esac 


The case construct is the shell scripting analog to switch in C/C++. It permits branching to one of a 
number of code blocks, depending on condition tests. It serves as a kind of shorthand for multiple 


if/then/else statements and is an appropriate tool for creating menus. 
case "$variable" in 


"$condition1" ) 
command... 


EE 


"$condition2" ) 
command... 


EE 


esac 


9 Quoting the variables is not mandatory, since word splitting does not take 


place. 
Ò Each test line ends with a right paren ). 
9 Each condition block ends with a double semicolon ;;. 


9 If a condition tests true, then the associated commands execute and the case 


block terminates. 


9 The entire case block ends with an esac (case spelled backwards). 


Example 11-24. Using case 


f!/bin/bash 
# Testing ranges of characters. 


echo; echo "Hit a key, then hit return." 
read Keypress 


case "SKeypress" in 
[[:lower:]] ) echo "Lowercase letter";; 
[[:upper:]] ) echo "Uppercase letter";; 
[91 ) Seine "ugmisp 
* 


) echo "Punctuation, whitespace, or other";; 
esac # Allows ranges of characters in [square brackets], 


# In the first version of this example, 


#+ [a-z] and [A-Z]. 


(go). =] (ox) Or) ES Ge) [Sy [——- Gy Wer (69) «cp (ox; (Gn des “Gor INS [E 


#+ or POSIX ranges in [[double square brackets. 


#+ the tests for lowercase and uppercase characters were 


# This no longer works in certain locales and/or Linux distros. 


19 
20 
Zi 
22 
23 
24 
25 
26 
27 
28 
29 


POSIX is more portable. 


Exercise: 


As the script stands, 


+ reports on each keystroke, 


it accepts a singl 
Change the script so it accepts repeated input, 


Thanks to Frank Wang for pointing this out. 


then terminates. 


keystroke, 


and terminates only when "X" is hit. 


Binat s 


enclose everything in a "while" loop. 


ex 


ie () 


Example 11-25. Creating menus using case 


(99) s dox, (On dex GOS) [A SS We) Co c] @) Gil de (63) RS [5 


47 


#!/bin/bash 
# Crude address database 
clear # Clear the screen. 
echo " Contact List" 
echo Pr 0 2 UE it Sy ARS Al Pah se res RT " 
echo "Choose one of the following persons:" 
echo 
echo "[E]vans, Roland" 
echo "[J]ones, Mildred" 
echo "[S]mith, Julie" 
echo "[Z]ane, Morris" 
echo 
read person 
case "$person" in 
# Note variable is quoted. 
"pn" | "o" ) 
# Accept upper or lowercase input. 
echo 
echo "Roland Evans" 
echo VASILE dedbeueor DE 
echo "Hardscrabble, CO 80753" 
echo W305) VOAL Aw 
eco “Y( SOS) 754—989 seo” 
echo "revans@zzy.net" 
echo "Business partner & old friend" 
Pr 
# Note double semicolon to terminate each option. 
gp | Poss ) 
echo 
echo "Mildred Jones" 
cechom WAS) m. TAB Stesp N9uo Jue 
echo "New York, NY 10009" 
echo: i212)" 533-231 4% 
echo "TM Bade hax” 
echo "milliej@loisaida.com" 
echo "Ex-girlfriend" 
echo "Birthday: Feb. 11" 
Pr 
# Add info for Smith & Zane later. 


49 EN) 


50 # Default option. 

Gl # Empty input (hitting RETURN) fits here, too. 
52 echo 

S9] echo "Not yet in database." 
54 P 

55 

56 esac 

57 

58 echo 

59 

60 # Exercise: 

Gil ge SS 


62 # Change the script so it accepts multiple inputs, 

63 #+ instead of terminating after displaying just one address. 
64 

65 exit 0 


An exceptionally clever use of case involves testing for command-line parameters. 


1 #! /bin/bash 

2 

3 case WS sug 

4 "") echo "Usage: ${0##*/} <filename>"; exit $E PARAM;; 

5 No command-line parameters, 

6 or first parameter empty. 

7 4 Note that ${0##*/} is S${var##pattern} param substitution. 

8 Net result is $0. 

9 

0 —*) FILENAME-./$1;; # If filename passed as argument ($1) 
i T GWEWEiEE Waitin wr Clasia, 

2 + replace it with ./$1 

3 * so further commands don't interpret it 
4 T as anm 09r Omn, 
15 
16 2) PIEBENAMESSL >} # Otherwise, $1. 
17 esac 


Here is an more straightforward example of command-line parameter handling: 


1 #! /bin/bash 

2 

3 

A warte || Si see ( ly «clo # Until you run out of parameters 
5 Case Ue cba 

6 Ecce) 

7 # "-d" or "--debug" parameter? 

8 DEBUG-1 

9 Pi 

10 -epe-compb 

al CONFFILE-"$2" 

2 lestie) 

3 if [ ! -f SCONFFILE ]; then 

4 echo "Error: Supplied file doesn't exist!" 
5 exit $E CONFFILE # File not found error. 
6 FA 

7 Pi 

8 esac 

Ig) Snbtistate # Check next set of parameters. 
20 done 


24l 

22 # From Stefano Falsetto's "Log2Rot" script, 
28) Gir jouer, «gue. euis. Woe e loe acka 

24 # Used with permission. 


Example 11-26. Using command substitution to generate the case variable 


[Es 
© Wey (or c rex; Gal des mos [eS p 


S 
= 


ere 
Wn 


#!/bin/bash 
# case-cmd.sh: Using command substitution to generate a "case" variable. 


case $( arch ) in # "arch" returns machine architecture. 
# Equivalent to 'uname -m' 


1386 ) echo "80386-based machine";; 
1486 ) echo "80486-based machine";; 
i586 ) echo "Pentium-based machine";; 
i686 ) echo "Pentium2+-based machine";; 
es ) echo "Other type of machine";; 
esac 
ese 0) 


A case construct can filter strings for globbing patterns. 


Example 11-27. Simple string matching 


NPRPRPRPRP PPP PY 
O io 00 -1 O Oi i& (0. M. I. O xo 0 -1 O O1 iS CQ) I ES 


(SS) DSS) [ser [mS DINI 
=] ce», (Gil GS (9 fe» [9 


N 
© 


wW CON G O w 193) (G3). (69) ESI 
=] oy (Ou dES (S [e$ [S ce» we 


#!/bin/bash 
# match-string.sh: Simple string matching. 


match_string () 

{ PAEICE SICISILING) mece, 

[ATCH=0 

E_NOMATCH=90 

ARAMS=2 # Function requires 2 arguments. 
E BAD PARAMS-91 


as) 


[ $4 -eq SPARAMS ] || return $E BAD PARAMS 


case "$1" in 
USA CU CAR MATECH P 


* ) return $E NOMATICH;; 
esac 
} 
a=one 
b=two 
c=three 
d=two 
match string $a wrong number of parameters 
echo $? gI 
maten string Sa S no match 
eco S9 90 
match string Sb Sd match 
echo $? 0 
exit 0 


Example 11-28. Checking for alphabetic input 


#!/bin/bash 
# isalpha.sh: Using a "case" structure to filter a string. 


SUCCESSO 


FAILURE=-1 
isalpha () # Tests whether *first character* of input string is alphabetic. 
( 
ase quoc: WS] # No argument passed? 
then 
return SFAILURE 
ial 


case "$1" in 


NPRPRPRP RPP PP PY 
O (o 0» -1 O Oi i$ (). NM. I. O xo 0 -1 O O1 4 CQ I 


lamna- Aley certa SSUCCINSS 22 ap Becas vica a Ikeieieeue? 
A ) earten SALGUD P 

esac 

} # Compare this with "isalpha ()" function in C. 
24L stel (0) # Tests whether *entire string* is alphabetic. 
22 4 
2:9 [ $$ -eg 1 ] || return SFAILURE 
24 
29 case $1 in 
26 e püewAo—2]5]WHUy senewuso GIEZXICIUISISO P 
27) S ertai SSUCCISS ps P 
28 esac 
29 } 
30 
SiL sbexehbbeplie (0 # Tests whether *entire string* is numerical. 
SE # In other words, tests for integer variable. 
33 [ $4 -eq 1 ] || return S$FAILURE 
34 
35 case $1 in 
36 *[!0-9]*|"") return SFAILURE;; 
37 **). smewemuem  SYISIUÍCICISEISYS E P 
Sis) esac 
39 } 
40 
41 
42 
43 check var () # Front-end to isalpha (). 
44 ( 
AS ai sisjsimer US 
46 then 
47 EEn WG becas vosbielar veo: edane (loveneracieyeie, Y 
48 34e Sieeudljeleyau2 whey 
49 then # No point in testing if first char is non-alpha. 
50 echo "\"S*\" contains only alpha characters." 
S else 
52 acho WAS Guyana at laast one ineim—euljolie, Character, V 
5/3] ít 
54 else 
55 echo "\"S*\" begins with a non-alpha character." 
56 # Also "non-alpha" if no argument passed. 
Gyy YE 
58 
59 echo 
60 


Oo 
[Em 
— 


select 


62 
63 
64 
65 
66 
67 
68 
69 
70 
Ga 
72 
73 
74 
15 
76 
T3 
78 
79 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
2t 
92 
9S 
94 
95 
96 
2i 
98 
9g, 
100 
Lal 
102 
119.3) 
104 
105 


The select construct, adopted from the Korn Shell, is yet another tool for building menus. 


digit check () # Front-end to isdigit (). 
( 
aft 3ieyohlgpii, "SU 
then 
geko AESA Ccomiamims (xw chigales [0 = 91." 
else 
echo "\"S*\" has at least one non-digit character." 
ita 


echo 


a-23skidoo 

b-H31lo 

c--What? 

d-What? 

e-'echo $b' # Command substitution. 
f-AbcDef 

g-27234 

h=27a34 

i-27.34 


Check var Sei 
check var $b 
check var Se 
check var se 
Check vai Se 
check var $f 
check var # No argument passed, so what happens? 
# 

digit check $g 
Ghleplic_cloeelhe Sia 
digitecheck sa 


exit 0 # Script improved by S.C. 


# Exercise: 


# Write an 'isfloat ()' function that tests for floating point numbers. 


i; lalimes Hae itiacicLem cial eceres Viselueae (0 ', 
#+ but adds a test for a mandatory decimal point. 


select variable [in list] 


do 


command... 


break 


done 


This prompts the user to enter one of the choices presented in the variable list. Note that select uses 


the $PS3 prompt (#? ) by default, but this may be changed. 


Example 11-29. Creating menus using select 


i 
2 


#!/bin/bash 


NO | H H H H H H H [- f 
O Wey (oer =] yy Gal dex GO IS) [^ (e Wey (ey —] cx) Gal des GS 


NNN PO 
dex d» fm» 


PS3='Choose your favorite vegetable: ' # Sets the prompt string. 
# Otherwise it defaults to #? 


echo 


select vegetable in "beans" "carrots" "potatoes" "onions" "rutabagas" 
do 

echo 

echo "Your favorite veggie is $vegetable." 

eto Yuck! 

echo 

break # What happens if there is no 'break' here? 
done 


exit 


# Exercise: 


# Fix this script to accept user input not specified in 

#+ the "select" statement. 

# For example, if the user inputs "peas," 

#+ the script would respond "Sorry. That is not on the menu." 


Ifin list is omitted, then select uses the list of command line arguments ($@) passed to the script 
or the function containing the select construct. 


Compare this to the behavior of a 


for variable [in list] 


construct with the in list omitted. 


Example 11-30. Creating menus using select in a function 


Re [m3 [R3 Mi gp ee eee Rees 
WVR c9» (er ee -— ey) (ab PS (eS) [my [5 65 (Wer (09) cp ep (Oa? ASS (es) iy [5 


N 
A 


#!/bin/bash 
PS3='Choose your favorite vegetable: ' 
echo 


choice. of() 
( 
select vegetabl 
# [in list] omitted, so 'select' uses arguments passed to function. 
do 
echo 
echo "Your favorite veggie is $vegetable." 
echo "Yuck!" 
echo 
break 
done 


) 


choice of beans rice carrots radishes tomatoes spinach 


# $1 $2 $3 $4 $5 $6 
# passed to choice_of() function 
esac 0) 


See also Example 36-3. 
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Chapter 12. Command Substitution 


Command substitution reassigns the output of a command [1] or even multiple commands; it literally plugs 
the command output into another context. [2] 


The classic form of command substitution uses backquotes C ..^). Commands within backquotes (backticks) 
generate command-line text. 


ll 
2 


Script name-'basename $0 
echo "The name of this script is S$script name." 


The output of commands can be used as arguments to another command, to set a variable, and even for 
generating the argument list in a for loop. 


g 


(s; cp fexy (ap des (Gs) SS) [ Sy Wey Cer cp cx) (Ga de G3 JR» [S 


«o 


rm “cat filename” # "filename" contains a list of files to delete. 
# 

Ww So Co polars Owe trac Varco ligte too lome emro wulcinc seul. 

# Better is xargs rm —- « filename 

# ( -- covers those cases where "filename" begins with a "-" ) 


textfile listing-' ls *.txt' 
# Variable contains names of all *.txt files in current working directory. 
cho $textfile listing 


textfile listing2-S$(ls *.txt) # The alternative form of command substitution. 
cho $textfile listing2 
Same result. 


A possible problem with putting a list of files into a single string 
is that a newline may creep in. 


A safer way to assign a list of files to a parameter is with an array. 
shopt -s nullglob # If no match, filename expands to nothing. 
textfile listing-( *.txt ) 


Thenkss S.C. 


Command substitution invokes a subshell. 


® Command substitution may result in word splitting. 


1 COMMAND "echo a b` # 2 args: a and b 
2 

3 COMMAND Hecho a b^" arog: ab 

4 

5 COMMAND * echo” # no arg 

6 

7 COMMAND "'echo'" # one empty arg 

8 

E 
JL() 47 ‘Woeunks, So. 


Even when there is no word splitting, command substitution can remove trailing newlines. 


# cd ""pwd'" # This should always work. 
# However... 


mkdir 'dir with trailing newline 
' 


cd 'dir with trailing newline 
' 


Wey (eer s ren) (np des (63) jest [3 


10 cd ""pwd'" 4$ Error message: 
11 # bash: cd: /tmp/file with trailing newline: No such file or directory 
12 


13 «yel USEN # Works fine. 

14 

LS 

16 

iy 

18 

19 old_tty_setting=$(stty -g) # Save old terminal setting. 

20 echo "Hit a key " 

All Gib icanon -Eche # Disable "canonical" mode for terminal. 
22 # Also, disable *local* echo. 

23 key=$(dd bs-1 count-1 2» /dev/null) 4 Using 'dd' to get a keypress. 
2A gis polec (iw settime” # Restore old setting. 

25 echo "You hit S{#key} key." # S{#variable} = number of characters in $variable 
26 

27 Hit any key except RETURN, and the output is "You hit 1 key." 

28 SE IRWIN, uso] 3b "m Yow boie O likey.” 


29 The newline gets eaten in the command substitution. 
30 
31 #Code snippet by Stéphane Chazelas. 


d Using echo to output an unquoted variable set with command substitution removes trailing newlines 


characters from the output of the reassigned command(s). This can cause unpleasant surprises. 


Gir lisitimg=" lg =~ 
echo $dir listing # unquoted 


# Expecting a nicely ordered directory listing. 


+ 


However, what you get is: 
coral 3 -mw ew e= 1 lowe looo 310 Mey 1S I 7915 Js -ewn e= 1 aoza 
bozo Sil May 15 20557 t2.8la -—rwkur—zuc-sx L bozo boza 217 Mer 5 2ls13 wi Sh 


+ 


# The newlines disappeared. 


PRR 
N P) O00 -10 O4 0 N p| 
+ 


13 ego "Sg Ia sie aio! # quoted 

14 4 -rw-rw-r-- 1 bozo 30 Mew IS 17215 L tsa 
I5 ap rw rw JL Joxoyuo Sl May 1S 20857 12.sm 
16 i SWE dL boza Ally Mew 5 Zilgils wal. ela 


Command substitution even permits setting a variable to the contents of a file, using either redirection or the 
cat command. 


Cs) =] fy) (Out des jG) [m3 [5 


variablel= «filel' 
variable2= cat filez 


echo 


wE | 


See '"AweucibedodheL' ico Comcenes (us Wienke. 
Sac "wecibedoie2 co Gomeemes or Verla, 
This, however, forks a new process, 
+ so the line of cod xecutes slower than the above version. 


Se SE ESE 


Note that the variables may contain embedded whitespace, 
+ or even (horrors), control characters. 


It is not necessary to explicitly assign a variable. 


0^ S Uu # Echoes the script itself to stdout. 


Excerpts from system file, /etc/rc.d/rc.sysinit 
+ (on a Red Hat Linux installation) 


SH /CSSCKOOELOGMS ]p then 
fsckoptions-' cat /fsckoptions" 


(sS; cp xy (Oa des (9 [S [jS €» wo 


[ -e "/proc/ide/${disk[$device]}/media" ] ; then 
hdmedia= cat /proc/ide/S$[(disk[$device])/media' 


[ | -a Uode -e | geep == P-E jy then 
Rtacg=" eat Proc Version | 


[ Susb = "1" ]; then 

sleep 5 

mouseoutput= cat /proc/bus/usb/devices 2>/dev/null|grep -E "^I.*Cls-03.*Prot-02"' 
kbdoutput-'cat /proc/bus/usb/devices 2>/dev/null|grep E "*I.*Cls=03.*Prot=01"" 


Do not set a variable to the contents of a /ong text file unless you have a very good reason for doing so. 
Do not set a variable to the contents of a binary file, even as a joke. 


Example 12-1. Stupid script tricks 


PRPrPPrPrRPP RB 
-] OY) UO) 4 (Q0 M) PP. O o 0 1 O Oi 45 €) N ES 


18 
19 


#!/bin/bash 
iy SEWMOLC—SeirIjSeAceLelks Silas Donie Ey chis ar lems, olks. 
i; wom USES) Serio Tricka, Y volue I - 


dangerous, variable-' cat /boot/vmlinuz' # The compressed Linux kernel itself. 
echo "string-length of N$dangerous variable = $(4dangerous variable)" 
string-length of $dangerous variable = 794151 

(Does not give same count as 'wc -c /boot/vmlinuz'.) 

echo "$dangerous variable" 


Dems tay tnig Te woulllel hanc tre Seis c 


The document author is aware of no useful applications for 
+ setting a variable to the contents of a binary file. 


eie 0) 


Notice that a buffer overrun does not occur. This is one instance where an interpreted language, such as 
Bash, provides more protection from programmer mistakes than a compiled language. 


Command substitution permits setting a variable to the output of a loop. The key to this is grabbing the output 
of an echo command within the loop. 


Example 12-2. Generating a variable from a loop 


#!/bin/bash 
# csubloop.sh: Setting a variable to the output of a loop. 


do 


vaciablel= for a aim L 2 3 4 5 


aca =i WE 7 Ug VeClva! eomiene! is Crit eall 


7 done’ #+ to command substitution here. 
8 

9 echo "variablel = $variablel" # variablel = 12345 
10 
db Ti 
12 i=0 
13 variable2-' while [ "$i" -1t 10 ] 
14 do 
15 CCHOM i SN # Again, the necessary 'echo'. 
16 ere Wal we dp # Increment. 

17 done’ 

18 

19 echo "variable2 = $variable2" # variable2 = 0123456789 
20 
21 # Demonstrates that it's possible to embed a loop 
22 #+ within a variable declaration. 
2/8 
24 sui O 


Command substitution makes it possible to extend the toolset available to Bash. It is simply a matter of 
writing a program or script that outputs to stdout (like a well-behaved UNIX tool should) and assigning 
that output to a variable. 


#include <stdio.h> 


ye hieme. worelel, (6 piocann 


il 
2 
3 
4 
5 lie AY 
G i 
7 iousuewEiE(, "use. veu wa Jy 
8 return (0); 
9 p 
bash$ gcc -o hello hello.c 


1 #!/bin/bash 

2 # hello.sh 

3 

4 greeting-'./hello^ 
5 echo $greeting 


bash$ sh hello.sh 
Hello, world. 


ə The $(...) form has superseded backticks for command substitution. 


GQuwlicowie=S (seci =i (VSI SELS) wr regu "ene. Sila! example. 


File contentsl-$(cat $filel) 
File contents2-$ (<$file2) # Bash permits this also. 


The $(...) form of command substitution treats a double backslash in a different way than ^... 


i 
2 
3 4 Setting a variable to the contents of a text file. 
4 
5 


bash$ echo ‘echo NV 


bash$ echo $(echo NX) 
N 


The $(...) form of command substitution permits nesting. [3] 


1 word count-$( wc -w $(echo * | awk '(print $8}"') ) 


Or, for something a bit more elaborate . . . 


Example 12-3. Finding anagrams 


!/bin/bash 
agram2.sh 
Example of nested command substitution. 


Uses "anagram" utility 

+ that is part of the author's "yawl" word list package. 
meras // // aLloyatloIL sie) s orooro timez Jost Tos yaw l-0, S32 c egets s rog 
http://bash.neuralshortcircuit.com/yawl-0.3.2.tar.gz 


PRPPPPRPRPRPHEE 
XO 00 —1 OY O1 4 (0 I) LPS. O (o O0 -1 Oy O1 4 Q0 I9 HS 


E NOARGS-66 
E BADARG-67 
MINLEN=7 
aie | oem SIP q 
then 
echo "Usage $0 LETTERSET" 
exit $E NOARGS # Script needs a command-line argument. 
elif [ ${#1} -lt SMINLEN ] 
then 
20 echo "Argument must have at least SMINLEN letters." 
2 exit $E BADARG 
29 EL 
2:9) 
24 
25 
AG IWIN E BG 6 $ # Must have at least 7 letters. 
27 # 1234567 
28 Anagrams-( $(echo $(anagram $1 | grep SFILTER) ) ) 
29 # SK $( nested command sub. ) J 
30 4 ( array assignment ) 
EI 
32 


33 echo "S${#Anagrams[*]} 7+ letter anagrams found" 

34 echo 

35 echo ${Anagrams [0] } # First anagram. 

36 echo ${Anagrams[1] } # Second anagram. 

er TECO: 

38 

39 # echo "${Anagrams[*]}" # To list all the anagrams in a single line 
40 

41 # Look ahead to the "Arrays" chapter for enlightenment on 

42 #+ what's going on here. 

43 

44 # See also the agram.sh script for an example of anagram finding. 
45 

46 exit $? 


Examples of command substitution in shell scripts: 


1. Example 11-7 
2. Example 11-26 
3. Example 9-16 
4. Example 16-3 
5. Example 16-22 


6. Example 16-17 
7. Example 16-54 
8. Example 11-13 
9. Example 11-10 
10. Example 16-32 
11. Example 20-8 
12. Example A-16 
13. Example 29-3 
14. Example 16-47 
15. Example 16-48 
16. Example 16-49 


Notes 


[1] For purposes of command substitution, a command may be an external system command, an internal 
scripting builtin, or even a script function. 


[2] Ina more technically correct sense, command substitution extracts the stdout of a command, then 
assigns it to a variable using the — operator. 


[3] In fact, nesting with backticks is also possible, but only by escaping the inner backticks, as John Default 


points out. 
dL wore corme: we w \ scho = | awk "exesboue SUNT 
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Chapter 13. Arithmetic Expansion 


Arithmetic expansion provides a powerful tool for performing (integer) arithmetic operations in scripts. 
Translating a string into a numerical expression is relatively straightforward using backticks, double 
parentheses, or let. 


Variations 
Arithmetic expansion with backticks (often used in conjunction with expr) 
1 z-expr $z + 3^ # The 'expr' command performs the expansion. 


Arithmetic expansion with double parentheses, and using let 
The use of backticks (backquotes) in arithmetic expansion has been superseded by double parentheses 


-- ((...)) and$((...)) -- and also by the very convenient let construction. 
dL 25 (E23) ) 
2 meg ( (ze) ) # Also correct. 
3 # Within double parentheses, 
4 #+ parameter dereferencing 
5 #+ is optional. 
6 
7 # S((EXPRESSION)) is arithmetic expansion. # Not to be confused with 
8 #+ command substitution. 
9 
10 
iit 
12 # You may also use operations within double parentheses without assignment. 
13 
14 n=0 
45 echo "n= Sm" # n= 0 
16 
17 CO t= ))) # Increment. 
19 4 (( Sm = 1 )) ne incorrect! 
19 echo "n = $n" #n=1 
20 
21. 
22 let z=z+3 
23 let "z += 3" # Quotes permit the use of spaces in variable assignment. 
24 # The 'let' operator actually performs arithmetic evaluation, 
25 #+ rather than expansion. 


Examples of arithmetic expansion in scripts: 


1. Example 16-9 
2. Example 11-14 
3. Example 27-1 
4. Example 27-11 
5. Example A-16 
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Chapter 14. Recess Time 


This bizarre little intermission gives the reader a chance to relax and maybe laugh a bit. 


Fellow Linux user, greetings! You are reading something which 
will bring you luck and good fortune. Just e-mail a copy of 

this document to 10 of your friends. Before making the copies, 
send a 100-line Bash script to the first person on the list 

at the bottom of this letter. Then delete their name and add 
yours to the bottom of the list. 


Don't break the chain! Make the copies within 48 hours. 
Wilfred P. of Brooklyn failed to send out his ten copies and 
woke the next morning to find his job description changed 

to "COBOL programmer." Howard L. of Newport News sent 
out his ten copies and within a month had enough hardware 
to build a 100-node Beowulf cluster dedicated to playing 
Tuxracer. Amelia V. of Chicago laughed at this letter 

and broke the chain. Shortly thereafter, a fire broke out 

in her terminal and she now spends her days writing 
documentation for MS Windows. 


Don't break the chain! Send out your ten copies today! 


Courtesy 'NIX "fortune cookies", with some alterations and many apologies 
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Part 4. Commands 


Mastering the commands on your Linux machine is an indispensable prelude to writing effective shell scripts. 
This section covers the following commands: 


e . (See also source) 

* àc 

* adduser 

* agett 

* agre 

* ar 

* arch 

eat 

e autoload 

e awk (See also Using awk for math operations) 

* badblocks 

* banner 

* basename 

* batch 

e bc 

e bg 

* bind 

e bison 

* builtin 

* bzgrep 

* bzip2 

* cal 

* caller 

* cat 
cd 
chattr 
chfn 
chgrp 
chkconfig 


e © © e 09 9 9 © 9 9 0 06 @ 
[el 
= 
o 
= 
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* compgen 
* complete 
* compress 
* coproc 


e cp 

* cpio 

* cron 

e crypt 

* csplit 

* cu 

e cut 

* date 

* dc 

e dd 

e debugfs 

* declare 

* depmod 

e df 

e dialog 

e diff 

e diff3 

e diffstat 

e dig 

e dirname 

e dirs 
disown 
dmesg 
doexec 
dos2unix 


e. 
e. 
e 
e. 
e. 
e. 
e. 
e. 
e. 
e. 
e 
e. 
e. 
e. 
e. 
* exec 
* exit (Related topic: exit status) 
* expand 
e 
e. 
e. 
e. 
e. 
e. 
e 
e. 
e 
e. 
e. 
e. 
e. 
e. 
e. 


export 
expr 


factor 
false 
fdformat 
fdisk 


* free 

e fsck 

e ftp 

o fuser 

e cetfacl 

* getopt 

* getopts 

* cettext 

* cett 

e gnome-mount 
* grep 

* oroff 

* sroupmod 
* groups (Related topic: the SGROUPS variable) 
e gs 

* O71 

* halt 

* hash 

* hdparm 

* head 

* help 

* hexdump 
* host 

* hostid 

* hostname (Related topic: the SHOSTNAME variable) 
* hwclock 
e iconv 

* id (Related topic: the SUID variable) 
e ifconfi 

e info 

e infocm 

© init 

e insmod 

e install 

e ip 

* ipcalc 

e iwconfi 

e jobs 

e join 

e jot 

e kill 

e killall 

* last 

* lastcomm 
* lastlog 

e Idd 

* less 

e let 

e lex 

e lid 

* In 

* locate 

e lockfile 


* logger 


* logname 
* logout 

* logrotate 
* look 

* losetup 
ME 

els 

e Isdev 


* mailstats 

* mailto 

e make 

* MAKEDEV 
* man 

e mapfile 

* mcookie 

e md5sum 

* merge 

* mes 

* mimencode 
e mkbootdisk 
e mkdir 

e mke2fs 

e mkfifo 

e mkisofs 

e mknod 

e mkswap 


e mktemp 
* mmencode 


* modinfo 
* modprobe 
* more 

* mount 

e msgfmt 
e mv 

enc 

* netconfi 
* netstat 

* new 

* nice 
enl 

* nm 

* nma 

* nohu 

* nslooku 


© objdum 
* od 

* openssl 
* passwd 
* paste 

e patch (Related topic: diff) 
e pathchk 
e pax 

* pgrep 

* pidof 

e ping 

* pkill 

* popd 
epr 

* printenv 
* printf 

* procinfo 
° ps 

* pstree 

e pix 

* pushd 

e pwd (Related topic: the $PWD variable) 
e. quota 

* rcp 

e rdev 

* rdist 

* read 
readelf 
readlink 
readonly 
reboot 
recode 
renice 
reset 
resize 
restore 
rev 
rlogin 
m 
rmdir 
rmmod 
* route 

* rpm 

* rpm2cpio 
e rsh 

* rsync 

e runlevel 
* run-parts 
erx 

* IZ 

* sar 

* scp 

* script 

* sdiff 


* sed 

e seq 

* service 

* set 

* setfacl 

* setquota 

* setserial 

* setterm 

e shalsum 

e shar 

* shopt 

e shred 

e shutdown 

* size 

e skill 

* slee 

* slocate 

* snice 
sort 
source 


swapoff 


swapon 
SX 


e e © 09 09e 9 0 09 9 9 09 9 9 08 09 9 9 09 9 9 09 9 9 o0 0. o9 o 
Wel 
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telinit 
e telnet 
e Tex 
* texexec 
* time 
e times 
e tmpwatch 
* top 
e touch 


e tput 


et 

* traceroute 
> true 

e tset 

e tsort 

e tty 

e tune2fs 

e type 

* typeset 

e ulimit 

* umask 

* umount 

* uname 

* unarc 

* unarj 

* uncompress 
* unexpand 
e uniq 

* units 

* unlzma 

* unrar 

* unset 


e © © 09 09 9 09 09e 9 9 © 9 9 9o 09 9 9 0o 99 © © @ @ 
c 
c 
C 
5 
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o 
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e yes 
* zcat 
e zdiff 
e zdum 
* 7egre 
e 7fere 
* zore 


e zip 
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Chapter 15. Internal Commands and Builtins 


A builtin is a command contained within the Bash tool set, literally built in. This is either for performance 
reasons -- builtins execute faster than external commands, which usually require forking off [1] a separate 
process -- or because a particular builtin needs direct access to the shell internals. 


When a command or the shell itself initiates (or spawns) a new subprocess to carry out a task, this is called 
forking. This new process is the child, and the process that forked it off is the parent. While the child 
process is doing its work, the parent process is still executing. 


Note that while a parent process gets the process ID of the child process, and can thus pass arguments to it, 
the reverse is not true. This can create problems that are subtle and hard to track down. 


Example 15-1. A script that spawns multiple instances of itself 


#!/bin/bash 
# spawn.sh 


PIDS-$(pidof sh $0) 4 Process IDs of the various instances of this script. 
P array-( $PIDS ) # Put them in an array (why?). 
echo SPIDS 4 Show process IDs of parent and child processes. 
let "instances = S(4P array[*]) - 1" # Count elements, less 1. 
# Why subtract 1? 
echo "Sinstances instance(s) of this script running." 
echo Est eoe echo 


sleep 1 # Wait. 
sh $0 # Play it again, Sam. 


exit 0 # Not necessary; script will never get to her 
# Why not? 


After exiting with a Ctl-C, 
+ do all the spawned instances of the script die? 
If so, why? 


JL 
2 
3 
4 
3 
6 
j 
8 
9 
10 
A 
12 
13 
14 
LS 
16 
17 
18 
19 
AQ) 
21 
22 
23 


N N 
ow 


Be CAKSIUL MO 1E Luin tois series Tto ION 
It will eventually eat up too many system resources. 


URED 
si = re» 


Is having a script spawn multiple instances of itself 
+ an advisable scripting technique. 
Why or why not? 


WW N 
[—. &) «e 


Generally, a Bash builtin does not fork a subprocess when it executes within a script. An external system 
command or filter in a script usually will fork a subprocess. 


A builtin may be a synonym to a system command of the same name, but Bash reimplements it internally. For 
example, the Bash echo command is not the same as /bin/echo, although their behavior is almost 
identical. 


#!/bin/bash 


i 

2 

3 Gein "ume line wise thes WUexelmoNU tomate dor. U 

4 /bin/echo "This line uses the /bin/echo system command." 


A keyword is a reserved word, token or operator. Keywords have a special meaning to the shell, and indeed 
are the building blocks of the shell's syntax. As examples, for, while, do, and ! are keywords. Similar to a 
builtin, a keyword is hard-coded into Bash, but unlike a builtin, a keyword is not in itself a command, but a 


subunit of a command construct. |2] 
TO 


echo 
prints (to stdout) an expression or variable (see Example 4-1). 


1 echo Hello 
2 echo $a 


An echo requires the -e option to print escaped characters. See Example 5-2. 


Normally, each echo command prints a terminal newline, but the —n option suppresses this. 


<p) An echo can be used to feed a sequence of commands down a pipe. 


1] 


i if echo YOWVNRY | Grap =G its # if [[ SVAR = *txt* 

2 then 

3 echo “SVAR contains the substring Sequence Yre YY 
4 de 


pe) An echo, in combination with command substitution can set a variable. 


a=`echo "HELLO" | tr A-Z a-z^ 


See also Example 16-22, Example 16-3, Example 16-47, and Example 16-48. 
Be aware that echo “command deletes any linefeeds that the output of command generates. 


The $IFS (internal field separator) variable normally contains Wn (linefeed) as one of its set of 
whitespace characters. Bash therefore splits the output of command at linefeeds into arguments to 


echo. Then echo outputs these arguments, separated by spaces. 


bash$ ls -1 /usr/share/apps/kjezz/sounds 


cue 3L roor COGE 1407 Nov 7 2000 reflect.au 
ZEW E poo dL root root 362 Nov 7 2000 seconds.au 


bash$ echo ^ls -1 /usr/share/apps/kjezz/sounds' 


Total AO) crunk- il woogie wood VLG Now 7 2000 iwerleeie . au 


So, how can we embed a linefeed within an echoed character string? 


Embedding a linefeed? 
cho "Why doesn't this string \n split on two lines?" 
DOSSiN VIE Spa. 


ak OA 


E 


Let's try something else. 


HAHAH i 0 PN EP 


echo 


EW G E I xeexou root 


printf 


echo $"A line of text containing 

a linefeed." 

# Prints as two distinct lines (embedded linefeed). 
# But, is the "$" variable prefix really necessary? 


[Es 


e 


echo 


ceho Vimus striing splits 
on two lines." 
# No, the "S$" is not needed. 


(sy - exp (ab des (09. (3) [5 ce) Wer (ee) 


e 


echo 
cho Ww Ww 
echo 


NNN NN 
GS ey f O ol 


N 
od 


echo -n $"Another line of text containing 

a linefeed." 

# Prints as two distinct lines (embedded linefeed). 

# Even the -n option fails to suppress the linefeed her 


N 
[91] 


NO PN 
-] Oo 


N 
[o9] 


echo 
echo 
cho " " 
echo 
echo 


w w N 
i &5 «e 


Ww 
N 


# However, the following doesn't work as expected. 
# Why not? Hint: Assignment to a variable. 
stringl=$"Yet another line of text containing 

a linefeed (maybe)." 


CO CO CO CO CO CO CO 
«oO 00 —-1 OY O1 fw 


echo $stringl 

# Yet another line of text containing a linefeed (maybe). 
n As 

# Linefeed becomes a space. 


PP b amÉ uS us 
(pb dE (3. [R3 [9 c0» 


# Thanks, Steve Parker, for pointing this out. 


$^) This command is a shell builtin, and not the same as /bin/echo, although its 
behavior 1s similar. 


bash$ type -a echo 
echo is a shell builtin 
echo is /bin/echo 


The printf, formatted print, command is an enhanced echo. It is a limited variant of the C language 
printf () library function, and its syntax is somewhat different. 


printf format-string... parameter... 


This is the Bash builtin version of the /bin/printf or /usr/bin/printf command. See the 
printf manpage (of the system command) for in-depth coverage. 


® Older versions of Bash may not support printf. 


Example 15-2. printf in action 


read 


PRP 


PRP 
YADUOBRWNPFPOOMIDUGURWNHE 


Boa 
O Xo oo 


CO) CO NO PO P9 PO PO IND PO PD NH 
i= &) We) (Ge) <a) fen Wal des (3 Iss) [S 
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#!/bin/bash 
# printf demo 


declare -r 
declare -r 


PI-3.14159265358979 
DecimalConstant-31373 


Messagel="Greetings," 


Message2="1 
echo 
joreatioiese WIP. 


echo 
prioci WIES 


aguciclall sine, 


# Read-only variable, i.e., a constant. 


io 2 cecimal places = Si ,.2ic Sir 


to 9 decimal places = $1.9f" SPI # It even rounds off correctly. 


jotne Y Vai” 


+ 


Prints a line feed, 
# Equivalent to 'echo' 


printf "Constant = \t%d\n" $DecimalConstant # Inserts tab (Nt). 


printf "£s $s \n" $Messagel $Message2 


echo 


# 


5 Saimlecieim OF © PUTEO, 
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# Loading a variable with a formatted string. 


echo 


Da LSS (owner VSI, zs qeu) 
echo "Pi te 12 decimal places = $P12" # Roundoff error! 


Msg= printf "£s %s \n" $Messagel SMessage2~ 


echo $Msg; 


echo $Msg 


# As it happens, the 'sprintf' function can now be accessed 
#+ as a loadable module to Bash, 
#+ but this is not portable. 


ewe d) 


Formatting error messages is a useful application of printf 


[Eh 
© We) OO zu ny Gal de» CH [ES | 


fies 
[n 


PRR 
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# Formats positional params passed, and sends them to stderr. 


exu S Cem"E exl tea Gs" 


E BADDIR-85 
var-nonexistent directory 
error() 
( 
primet MSE" Sez 
echo 
exit SE_BADDIR 
} 
cel Svar || || 
s tasir SoC, 


See also Example 35-15. 


" Svar" 


"Reads" the value of a variable from st din, that is, interactively fetches input from the keyboard. 
The —a option lets read get array variables (see Example 27-6). 


Example 15-3. Variable assignment, using read 


! /bin/bash 
"Reading" variables. 


cho -n "Enter the value of variable 'varl': " 
The -n option to echo suppresses newline. 


read varl 
Note no '$' in front of varl, since it is being set. 


echo "varl = $varl" 


echo 


# A single 'read' statement can set multiple variables. 
cho -n "Enter the values of variables 'var2' and 'var3' " 


cho -n "(separated by a space or tab): " 
read var2 var3 
echo "var2 = Svar2 var3 = Svar3" 


NPRPRPP PPP PP 
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# If you input only one value, 
#+ the other variable(s) will remain unset (null). 


NNN 
w N e 


exit 0 


A read without an associated variable assigns its input to the dedicated variable $REPLY. 


Example 15-4. What happens when read has no variable 


#!/bin/bash 
# read-novar.sh 


echo 


# # 
echo -n "Enter a value: " 

read var 

echo Wy Marea) U = UJ isses Ud " 


# Everything as expected here. 
# 

echo 
# T 
echo -n "Enter another value: " 
read # No variable supplied for 'read', therefore... 

#+ Input to 'read' assigned to default variable, SREPLY. 
var="SREPLY" 


NPRPRPP RPP PPP 
O (o 0» 1 O Ui i& (). M PS. O «o 0 -1 O O1 i Q I E 
se 


echo Bs pare = U Severa Ul " 
# This is equivalent to the first code block. 


N N 
N he 
=+ 
E 


25 

24 echo 

25 echo ™ n 

26 echo 

2 

28 

29 # This example is similar to the "reply.sh" script. 
30 4$ However, this one shows that SREPLY is available 


St 
32 
33 
34 
35 
36 
3 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 


#+ even after a 'read' to a variable in the conventional way. 


# In some instances, you might wish to discard the first value read. 
# In such cases, simply ignore the SREPLY variable. 


( 4 Code block. 
read # Line 1, to be discarded. 
read line2 # Line 2, saved in variable. 
) «$0 
Sle Vain 2) Oi ielaus sera ies 
echo "Sline2" # # read-novar.sh 
echo # #!/bin/bash line discarded. 
# See also the soundcard-on.sh script. 


eve O 


Normally, inputting a \ suppresses a newline during input to a read. The -r option causes an 
inputted \ to be interpreted literally. 


Example 15-5. Multi-line input to read 


NPRPRPRPRP PPP PY 
O (o 00 -1 O Oi i$ (). NM. IS. O xo 0 -1 O O1 i CQ I ES 


ISS) (sS SY [3X Ther Iss) eo) 
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#!/bin/bash 


echo 


echo "Enter a string terminated by a \\, then press <ENTER>." 
echo "Then, enter a second string (no \\ this time), and again press <ENTER>." 


# The "\" suppresses the newline, when reading $varl. 
# Eest Lins X 
# second line 


read varl 


echo "varl = $varl" 
# varl = first line second line 


# For each line terminated by a "\" 
#+ you get a prompt on the next line to continue feeding characters into varl. 


echo; echo 


echo "Enter another string terminated by a \\ , then press <ENTER>." 


read -r var2 # The -r option causes the "\" to be read literally. 
# fies lime \ 

echo "var2 = $var2" 

# var2 = first line \ 

# Data entry terminates with the first <ENTER>. 


echo 


exalic (0) 


The read command has some interesting options that permit echoing a prompt and even reading 
keystrokes without hitting ENTER. 


1 # Read a keypress without hitting ENTER. 

2 

3 read -s -nl -p "Hit a key " keypress 

4 echo; echo "Keypress was "\"Skeypress\""." 

5 

6 => option means do not echo input. 

7 -n N option means accept only N characters of input. 

8 -p option means echo the following prompt before reading input. 

9 
10 Using these options is tricky, since they need to be in the correct order. 

The -n option to read also allows detection of the arrow keys and certain of the other unusual keys. 


Example 15-6. Detecting the arrow keys 


#!/bin/bash 
# arrow-detect.sh: Detects the arrow keys, and a few more. 
# Thank you, Sandro Magi, for showing me how. 


# 
# Character codes generated by the keypresses. 
arrowup='\[A' 

arrowdown='\[B' 

arrowct=" C" 

arrowleft='\[D' 

diagewice="\ [LZ " 

delete='\[3' 


SUCCESS—0 
OTHER=65 


Glove, =i Wiis a Key oo  U 
# May need to also press ENTER if a key not listed above pressed. 


NPRPRPPRPP PPE 
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exit SSUCCESS 
i aL 


read -n3 key # Read 3 characters. 
23 
22 echo -n "Skey" | grep "Sarrowup" #Check if character code detected. 
23 ss || WS?9w ee SSUOONSS | 
24 then 
25 echo "Up-arrow key pressed." 
26 exit $SUCCESS 
2 Seal, 
28 
29 echo -n "Skey" | grep "Sarrowdown" 
30 asf [ "S9w eg SSUCCHSSS | 
31 then 
32 echo "Down-arrow key pressed." 
33 exit $SUCCESS 
3A Pi 
35 
36 echo -n "Skey" | grep "Sarrowrt" 
Sy ane | VSev ee SSUCOHNSS | 
38 then 
39 echo "Right-arrow key pressed." 
40 exit $SUCCESS 
Ad dex 
42 
43 echo -n "Skey" | grep "Sarrowleft" 
44 sx [ "$9" es SSUCCHSS ] 
45 then 
46 echo "Left-arrow key pressed." 
47 
48 


49 


50 echo -n "Skey" | grep "Sinsert" 
Gil atic | VSO" eer SSUCCESS | 

52 then 

53 echo "\"Insert\" key pressed." 


54 exit SSUCCESS 


$5 tu 

56 

57 exeo =a "Sikew" | greo "Selelss" 
5g sx | US9" es SSUCCESS ] 

59 then 

60 echo "\"Delete\" key pressed." 


61 exit $SUCCESS 

(92) dra 

63 

64 

65 echo " Some other key pressed." 
66 

67 exit SOTHER 

68 
69 # 
70 
71 # Mark Alexander came up with a simplified 
72 #+ version of the above script (Thank you!). 
73 # It eliminates the need for grep. 

74 

75 #!/bin/bash 

76 

73] uparrow-$'Nx1lb[A' 

78 downarrow-$'Nxlb[B' 

7g) leftarrow=$'\x1lb[D' 

80 rightarrow=$'\xl1b[C' 


81 

82 read -s -n3 -p "Hit an arrow key: " x 
83 

84 Gage "pU shin 

85 Suparrow) 

86 echo "You pressed up-arrow" 
87 p 

88 $downarrow) 

39) echo "You pressed down-arrow" 
90 8$ 

Sil Sleftarrow) 

92 echo "You pressed left-arrow" 
93 BE 

94 Srightarrow) 

925 echo "You pressed right-arrow" 
96 PE 

Oe. esac 

98 

90 cua GE 
100 
101 # 
LOZ 


103 # Antonio Macchi has a simpler alternativ 
104 
105 #!/bin/bash 


106 

107 while true 

108 do 

199 read -snl a 

LO Test WV" == "exe ia AS I xxu: 
LT read -sni a 

JE test "$a" == "[" || continue 


LES read -snl a 
114 Calsemt Sad ITI 


115 A) echo "up" 


JENG B) echo “down”: >? 

1317) (2)  eemo "seigiEU s P 

ils D) echo “befti™s; 

119 esac 

120 done 

1271 

122 # # 
11229] 

124 4 Exercise: 

125 gj —u————— 


126 # 1) Add detection of the "Home," "End," "PgUp," and "PgDn" keys. 


H) The -n option to read will not detect the ENTER (newline) key. 


The -t option to read permits timed input (see Example 9-4 and Example A-41). 


The —u option takes the file descriptor of the target file. 


The read command may also "read" its variable value from a file redirected to st din. If the file 
contains more than one line, only the first line is assigned to the variable. If read has more than one 
parameter, then each of these variables gets assigned a successive whitespace-delineated string. 
Caution! 


Example 15-7. Using read with file redirection 


#!/bin/bash 


read varl «data-file 
exeo. "wel = Seed 
# varl set to the entire first line of the input file "data-file" 


read var2 var3 «data-file 

echo "var2 = Svar2 vars = Svar3" 

# Note non-intuitive behavior of "read" here. 

1) Rewinds back to the beginning of input file. 

2) Each variable is now set to a corresponding string, 
separated by whitespace, rather than to an entire line of text. 

3) The final variable gets the remainder of the line. 

4) If there are more variables to be set than whitespace-terminated strings 
on the first line of the file, then th xcess variables remain empty. 


A 


m oH 


Ge 


Se SE che che cod 


(= [s 
=] ep) (wp des (93. [> [9 ee) We) (ex c ey (nl gEx (es) [sy [5 


chow 


# How to resolve the above problem with a loop: 
while read line 
do 
echomclemda 
done <data-file 
# Thanks, Heiner Steven for pointing this out. 
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# Use SIFS (Internal Field Separator variable) to split a line of input to 
# "read", if you do not want the default to be whitespace. 


CO CO hO 
i &9 Ge 


@elme Hile (oue ciL wisercss v 
OIFS-SIFS; IFS-: # /etc/passwd uses ":" for field separator. 
while read name passwd uid gid fullname ignore 


w w 
Wh 


34 do 

35 echo Vsneme (Sru lise) Y 

36 done </etc/passwd # I/O redirection. 

37 IFS=SOIFS # Restore original SIFS. 

38 4 This code snippet also by Heiner Steven. 

39 

40 

41 

42 # Setting the SIFS variable within the loop itself 


43 


44 #+ in a temporary variable. 

45 # Thanks, Dim Segebart, for pointing this out. 
46 echo " 

47 echo "List of all users:" 

48 

49 while IFS-: read name passwd uid gid fullname ignore 
50 do 

51 echo "$name ($fullname)" 

52 done «/etc/passwd # I/O redirection. 

53 

54 echo 

55 echo STIS stili Sis" 

56 

57 exit 0 


#+ eliminates the need for storing the original SIFS 


Piping output to a read, using echo to set variables will fail. 


Yet, piping the output of cat seems to work. 


OTe GO Nl ESX 


cat filel file2 | 
while read line 
do 

echo Sline 

done 


However, as Bjón Eriksson shows: 


Example 15-8. Problems reading from a pipe 


#!/bin/sh 
# readpipe.sh 
# This example contributed by Bjon 


last-" (null)" 
Gare OO | 
while read line 
do 
exco "ele 
last=Sline 
done 


echo 
echo "4r44ETEYLYAFTEEBEFY. AAA 
oeaiei WADA cone, lasts SINE alU 


exit 0 End of code. 


Eriksson. 


(Parelal) OWE SUE OIF scripte Jl louis 
The 'echo' supplies extra brackets. 


PRPRPPPRPRPRPEHP EB 
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N N 
= e» 
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22 

23 ./readpipe.sh 
24 

25 {#!/bin/sh} 

26 {last="(null)"} 


27 {eae SO |} 
28 {while read line} 
29 {do} 


30 {echo "{S$line}"} 
31 {last=Sline} 


32 {done} 

33 (primeri Viwlil come, dass Siem) 
34 

35 

36 All done, last: (null) 

Sy 


38 The variable (last) is set within the loop/subshell 
39 but its value does not persist outside the loop. 


The gendiff script, usually found in /usr/bin on many Linux distros, pipes the 
output of find to a while read construct. 


i eime Sil \( www USZV =o -Tana "S24 X) were | 
2 while read f; do 
3 
It is possible to paste text into the input field of a read (but not multiple lines!). See 


Example A-38. 


Filesystem 


cd 
The familiar ed change directory command finds use in scripts where execution of a command 
requires being in a specified directory. 


i (eel /souree/chimacromy && tar et = s ) || (eel /clesie/climectomy && tar xqowi =) 
[from the previously cited example by Alan Cox] 


The —P (physical) option to cd causes it to ignore symbolic links. 


cd - changes to SOLDPWD, the previous working directory. 


The ed command does not function as expected when presented with two forward 
slashes. 


bash$ ed // 
bash$ pwd 
fd 


The output should, of course, be /. This is a problem both from the command-line and 
in a script. 
pwd 
Print Working Directory. This gives the user's (or script's) current directory (see Example 15-9). The 
effect is identical to reading the value of the builtin variable SPWD. 
pushd, popd, dirs 
This command set is a mechanism for bookmarking working directories, a means of moving back and 
forth through directories in an orderly manner. A pushdown stack is used to keep track of directory 


names. Options allow various manipulations of the directory stack. 


pushd dir-name pushes the path di r-name onto the directory stack and simultaneously 
changes the current working directory to di r-name 


popd removes (pops) the top directory path name off the directory stack and simultaneously changes 
the current working directory to that directory popped from the stack. 


dirs lists the contents of the directory stack (compare this with the SDIRSTACK variable). A 
successful pushd or popd will automatically invoke dirs. 


Scripts that require various changes to the current working directory without hard-coding the 
directory name changes can make good use of these commands. Note that the implicit SDIRSTACK 
array variable, accessible from within a script, holds the contents of the directory stack. 


Example 15-9. Changing the current working directory 


#!/bin/bash 


dirl=/usr/local 
dir2=/var/spool 


pushd S$dirl 
ur Wali do gin aŭtonearcie “chest (diec chicory, stack cE siscloule)) c 
echo "Now in directory '"pwd'." # Uses back-quoted 'pwd'. 


# Now, do some stuff in directory 'dirl'. 
pushd $dir2 
echo "Now in directory '"pwd'." 


# Now, do some stuff in directory 'dir2'. 

echo "The top entry in the DIRSTACK array is $DIRSTACK." 
popd 

echo "Now back in directory '"pwd'." 


# Now, do some more stuff in directory 'dirl'. 
popd 


NPRPRPP PPP PP 
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21 echo "Now back in original working directory '"pwd'." 

22 

23 exit 0 

24 

25 # What happens if you don't 'popd' then exit the script? 


26 # Which directory do you end up in? Why? 


Variables 


let 
The let command carries out arithmetic operations on variables. [3] In many cases, it functions as a 
less complex version of expr. 


Example 15-10. Letting let do arithmetic. 


1 #!/bin/bash 
2 

3 echo 

4 


eval 


5 let a-11 Same as 'a-11' 
6 let a=at+5 Equivalent to let "aa -« 5" 
7 (Double quotes and spaces make it more readable.) 
8 echo hh = 5 Sa 16 
9 
OME lei eee SU Equivalent to let "a =a << 3" 
11 echo "\"\Sa\" (=16) left-shifted 3 places = $a" 
12 128 
TS 
14 let "a /= 4" Equivalent to let "a- a / 4" 
15 ceo WI29 / A = Sav 32 
16 
dA lee ECCE MU Equivalent ir diee Ya = e = Si! 
JL (eleg) VAS. = Gy E 5E 27 
19 
AO dete: Wer tees hen Equivalent to let "a =a * 10" 
Ail exeo. Wz IO) = Sew! DIO 
22 
29. Jeu "er es (eu Equivalent to let "a =a % 8" 
24 echo "270 modulo 8 = Sa (270 / 8 = S3 iaeumeaiigeleie Sei) " 
25 6 
26 
27 
28 4 Does "let" permit C-style operators? 
29 # Yes, just as the (( ... )) double-parentheses construct does. 
30 
Sil JlexE ure # C-style (post) increment. 
32 GENO lere = Sal! # 6042 7 
33 Jet a== # C-style decrement. 
34 Secho “Joa = Sm # 7-- = 6 
35 # Of course, tta etc., also allowed 
36 echo 
3 
38 
39 4 Trinary operator. 
40 
41 # Note that $a is 6, see above. 
AP S aea TA # True 
4i] 1eXOlckoy fo ae 
44 
45 let at+ 
d oret Wits eke 7 gale # False 
47 echo $t # ial 
48 
49 exit 


eval argl [arg2] ... [argN] 


Combines the arguments in an expression or list of expressions and evaluates them. Any variables 
within the expression are expanded. The net result is to convert a string into a command. 


į ) The eval command can be used for code generation from the command-line or within 
a script. 


bash$ command string-"ps ax" 
bash$ process="ps ax" 


bash$ eval "Scommand_string" | grep "Sprocess" 
26973 pts/3 RG 000 creo color jos ers 
26974 pts/3 R+ () 890). jos a 


Each invocation of eval forces a re-evaluation of its arguments. 


(cer P (x (ap dms Go WS) [3 


b-'S$c' 
c-d 
echo $a # Sb 

# First level. 
eval echo Sa # Sc 

# Second level. 
eval eval echo Sa dd 

# Third level. 


ur lüeveuaUk. YAOI, Ii, Cnoc a 


Example 15-11. Showing the effect of eval 


Mop p opp ppppGÀO: 
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#!/bin/bash 
# Exercising "eval" 


we eval Jue <1” Gf  igongetlewe d) y= dee mS 


echo $y #+ but linefeeds removed because "echoed" variable is unquoted. 
echo 
ceno WS #  Linefeeds preserved when variable is quoted. 


echo; echo 


y= eval df^ # Similar to y= df- 
echo $y #+ but linefeeds removed. 


# When LF's not preserved, it may make it easier to parse output, 
#+ using utilities such as "awk". 


echo 
cho Ww Ww 
echo 


eval ""seq 3 | sed -e 's/.*/echo var&-ABCDEFGHIJ/'^" 
# varl-ABCDEFGHIJ 
# var2=ABCDEFGHIJ 
# var3-ABCDEFGHIJ 


echo 
cho "n "n 
echo 


# Now, showing how to do something useful with "eval" 
# (Thank you, E. Choroba!) 


version-3.4 # Can we split the version into major and minor 
#+ part in one command? 
cho "version = $version" 
eval major=${version/./;minor=} # Replaces '.' in version by ';minor=' 


# The substitution yields '3; minor=4' 
#+ so eval does minor=4, major-3 
echo Major: $major, minor: $minor # Major: 3, minor: 4 


Example 15-12. Using eval to select among variables 


il 
2 


#!/bin/bash 
# arr-choice.sh 


# Passing arguments to a function to select 
#+ one particular variable out of a group. 


shee i) L3 ale aks) JN. by 4} 
eucde( 20 21 22 23 24 25.) 
Encu—1(. 99 Sik BA 319» 3A 35.) 
# LIE UMSO S eee a a Element number (zero-indexed) 


choose_array () 


{ 


eval array_member=\${arr${array_number} [element_number] } 


i ^ A^A^A^A^AAAAAAAA 


# Using eval to construct the name of a variable, 
#+ in this particular case, an array name. 


NO +t H H H H H H H i- d 
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echo "Element $element number of array $array number is $array member" 


21 } # Function can be rewritten to take parameters. 
22 

23 array number-0 IPILIESHE, Queis s 

24 element number-3 

25 choose array aM 

26 

27 array number-2 Third array. 

28 element number-4 

29 choose array 34 

30 

3l array number-3 Null array (arr3 not allocated). 
32 element number-4 

33 choose array enya 

34 


35 7 Moans wou, ADMONO MECClil, Or [game clas Owe 


Example 15-13. Echoing the command-line parameters 


#!/bin/bash 
# echo-params.sh 


# Call this script with a few command-line parameters. 
# For example: 


il 

2 

3 

4 

5 

6 # Gin exeleo-josursuüs. sli Tirst SeC@omel Cairo sPexsbeicim Pirtin 

7 

8 params=$# # Number of command-line parameters. 
9 param=1 # Start at first command-line param. 
10 

11 while [ "$param" le "Sparams" ] 

12 elg 

13 echo -n "Command-line parameter " 

14 echo n \S$Sparam Gives only the *name* of variable. 
15 3 SES Sil, 92: SS, Gees 

16 Why? 

17 \S esespes ime first US 

18 + so it echoes literally, 

19 + and $param dereferences "Sparam" 
20 


t... as expected. 
Dale Colom. sen 
22 eval echo \$$param Gives the *value* of variable. 
Aor EAA AA The "eval" forces the *evaluation* 
24 4b (oue \SS 
25 + as an indirect variable reference. 


N 
[oD] 


2 (( Berein x JY # On to the next. 
28 done 

29 

30 cg GS 

Bal 
928 
33 
34A Si Git (Xelexo -4oxeucteWns crede! rirse SSverouoyel Calre Se(oybhelEle. sessile 


35 Command-line parameter $1 = first 
36 Command-line parameter $2 = second 
37 Command-line parameter $3 = third 
38 Command-line parameter $4 = fourth 
39 Command-line parameter $5 = fifth 


Example 15-14. Forcing a log-off 


1 #!/bin/bash 

2 HELI ALSHiaKe; aJ CO force a Logori, 

3 For dialup connection, of course. 

4 

5 Script should be run as root user. 

6 

7 SERPORT-ttyS3 

8 Depending on the hardware and even the kernel version, 
9 #+ the modem port on your machine may be different 

T0 #+ /dev/ttyS1 or /dev/ttyS2. 

dal 

12 

13 J&xbiliisrmyoE-ewwedL kiil —9 "jog ex || awk "Jp 1 foresee Si, jV 
14 prhocessS ID OIE j9j9 —————--— 
1.5 

4G Sher ililjeyeys # This variable is now a command. 
1.7 

18 

19 # The following operations must be done as root user. 
20 
21 chmod 666 /dev/S$SERPORT # Restore r+tw permissions, or else what? 


22 # Since doing a SIGKILL on ppp changed the permissions on the serial port, 
23 #+ we restore permissions to previous state. 


25 rm /var/lock/LCK..$SERPORT # Remove the serial port lock file. Why? 


27 exit $? 


29 # Exercises: 


31 4$ 1) Have script check whether root user is invoking it. 
32 # 2) Do a check on whether the process to be killed 


OO is actually running before attempting to kill it. 
34 # 3) Write an alternate version of this script based on 'fuser': 
35 #+ if [ fuser -s /dev/modem ]; then 


Example 15-15. A version of rot13 


#!/bin/bash 
up JA version ose Viol” usino eval, 
# Compare to "rot13.sh" example. 


Onl gm. (o3 [Roy (5 


setvar rot 13() 4 "rot13" scrambling 


& 1 

7 local varname=$1 varvalue=$2 

8 eval $varname-'$(echo "S$varvalue" | tr a-z n-za-m)' 

9 p 

10 

LL 

ie seicwee roc l5 var Wireoloeue iv luum Vit@olosie throug rocl 
13 echo $var # sbbone 

14 

L5 setya cot 13 var Vyar” 4 Run "sbbone" through rot13. 
16 # Back to original variable. 
7 eeh Seu # foobar 

18 

19 # This example by Stephane Chazelas. 


20 # Modified by document author. 
A 
22 ew U 


The eval command occurs in the older version of indirect referencing. 


1 eval var-N$$var 


The eval command can be risky, and normally should be avoided when there exists a 
reasonable alternative. An eval SCOMMANDS executes the contents of COMMANDS, 
which may contain such unpleasant surprises as rm -rf *. Running an eval on 
unfamiliar code written by persons unknown is living dangerously. 
set 
The set command changes the value of internal script variables/options. One use for this is to toggle 
option flags which help determine the behavior of the script. Another application for it is to reset the 
positional parameters that a script sees as the result of a command (set ^ command" ). The script 
can then parse the fields of the command output. 


Example 15-16. Using set with positional parameters 


1 #!/bin/bash 

2 4 ex34.sh 

S s» Serius "exei 

4 

5 4 Invoke this script with three command-line parameters, 
6 # for example, "sh ex34.sh one two three". 

7 

8 echo 

9 echo "Positional parameters befor set \\ uname -a\`^ :" 
10 echo "Command-line argument #1 = $1" 

11 echo "Command-line argument #2 = $2" 

12 echo "Command-line argument #3 = $3" 

13 

14 

15 set "unam a 4 Sets the positional parameters to the output 
16 # of the command “uname -a` 

L7 

18 echo 

LQ SEINO siu 
20 echo $ # +++++ 
21 # Flags set in script. 
22 echo $ # hB 
23 # Anomalous behavior? 
24 echo 
25 
26 echo "Positional parameters after set \` uname -a\ù :" 


27 
28 
219 
30 
Sill 
22 
33 
34 
39 


# Sl, $2, $3, etc. reinitialized to result of "uname -a^ 


echo "Field #1 of ‘uname -a' = $1" 
echo "Field 42 of 'uname -a' - $2" 
echo "Field #3 of ‘uname -a' = $3" 
echo \#\#\# 

echo $_ LES III 

echo 

exi O 


More fun with positional parameters. 


Example 15-17. Reversing the positional parameters 


Mop p opp ppppÀ: 
O (o 0» -1 O Oi i$ (). M PB. O xo 0 -1 O O1 i CQ I 


IS) DSF TS) INS) TSS) Ror de» 
ss} (om. (Gal HES (9 IS) [3 
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#!/bin/bash 
# revposparams.sh: Reverse positional parameters. 
# Script by Dan Jacobson, with stylistic revisions by document author. 


Ser s is © e ep 


# a A Spaces escaped 

# MES Spaces not escaped 

OIFS-SIFS; IFS-:; 

# 2 Saving old IFS and setting new one. 
echo 


(unreal [P Sh; exer 0 | 


do # Step through positional parameters. 
echo "### kO = "$k"" # Before 
k=$1:$k; # Append each pos param to loop variable. 
# ^ 
Cobos M ST # After 
echo 
Sala wie, c 
done 


set Sk # Set new positional parameters. 


echo - 
echo $4 # Count of positional parameters. 
echo - 
echo 
ite aL w^ (QuiiEiEUO ehe Walim laste Sers rie wasuesbelgdle —— db —— 
#+ to the positional parameters. 
do 
echo $i # Display new positional parameters. 
done 


IFS=SOIFS # Restore IFS. 


Question: 
Is it necessary to set an new IFS, internal field separator, 
+ in order for this script to work properly? 
What happens if you don't? Try it. 
And, why use the new IFS a colon -= in line T7, 
+ to append to the loop variable? 
What is the purpose of this? 


Se ce cfe Gb ch SHE HE 


exatic (0) 


$ ./revposparams.sh 


50 
Sal 
52 
S 
54 
55 
56 
57 
58 
3$ 
60 
61 
62 
63 
64 
65 


k0 = 
=ab 

kO =a b 

k= ¢ a !9 

ed = e a g 

i9 = tell ty del ei de) 
3 
de 
[s 
ab 


Invoking set without any options or arguments simply lists all the environmental and other variables 
that have been initialized. 


bash$ set 
AUTHORCOPY-/home/bozo/posts 
BASH-/bin/bash 

BASH VERSION-$'2.05.8(1)-release' 


XAUTHORITY-/home/bozo/.Xauthority 
_=/Sre/sasinice 

variable22-abc 

variable23-xzy 


Using set with the —— option explicitly assigns the contents of a variable to the positional parameters. 
If no variable follows the —— it unsets the positional parameters. 


Example 15-18. Reassigning the positional parameters 


NOB pop ppppHppGÀ: 
O (o 0» -1 O Oi i& (). NN IP. O (o 0 -1 O O1 i Q I E 


[SY [eS [SY [mx H3» [e D 
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#!/bin/bash 
variable="one two three four five" 


set -- $variable 
# Sets positional parameters to the contents of "S$variable". 


first param-$1 

second param-$2 

Sluice SmaieiE # Shift past first two positional params. 
i? Sinise 2 also works. 

remaining params-"$*" 


echo 

echo "first parameter = $first param" # one 
echo "second parameter - $second param" # two 
echo "remaining parameters = $remaining params" # three four five 
echo; echo 

# Again. 

set -- $variable 

first_param=$1 

second param-$2 

echo "first parameter = $first param" # one 
echo "second parameter - $second param" # two 


unset 


export 


28 f 
29 
30) set == 

31 4$ Unsets positional parameters if no variable specified. 
32 

33 first param-$1 

34 second param-$2 


35 Cho Witiiesic parcneter = Siciiesic parcem # (null value) 
36 echo "second parameter = $second_param" # (null value) 
37 

38 exit 0 


See also Example 11-2 and Example 16-56. 


The unset command deletes a shell variable, effectively setting it to null. Note that this command 
does not affect positional parameters. 


bash$ unset PATH 
bash$ echo $PATH 


bash$ 


Example 15-19. "Unsetting"' a variable 


1 #!/bin/bash 

2 4 unset.sh: Unsetting a variable. 

3 

4 variable-hello imac sala wee. 

5 echo "variable = Svariable" 

6 

7 unset variable Unset. 

8 In this particular context, 
9 + same effect as: variable= 
10 echo "(unset) variable = Svariable" Svariable is null. 
ILL 
412 ae || —m Veweuciglede" | Try a string-length test. 
13 then 
14 echo "\Svariable has zero length." 
US dea 
16 
U7 emu 0 


8^; In most contexts, an undeclared variable and one that has been unset are equivalent. 
However, the ${parameter:-default} parameter substitution construct can distinguish 
between the two. 


The export [4] command makes available variables to all child processes of the running script or 
shell. One important use of the export command is in startup files, to initialize and make accessible 
environmental variables to subsequent user processes. 


d Unfortunately, there is no way to export variables back to the parent process, to the 
process that called or invoked the script or shell. 


Example 15-20. Using export to pass a variable to an embedded awk script 


1 #!/bin/bash 
2 
3 # Yet another version of the "column totaler" script (col-totaler.sh) 
4 ij.» Coat acle Wis) a &yexexesL:sUexe| column (Oe meMlosics)) im tie target Tile, 
5 # This uses the environment to pass a script variable to 'awk' 
& 3j ancl places the aws seript in a variable, 
7 
8 
9 ARGS-2 
10 E WRONGARGS-85 
iil 
12 if [ $4 -ne "SARGS" ] # Check for proper number of command-line args. 
13 then 
14 echo "Usage: 'basename $0' filename column-number" 
LS exit SE_WRONGARGS 
LG itu 
Ly 
18 filename-$1 
19 column number-$2 
20 
21 #===== Same as original script, up to this point =====# 
22 
23 export column number 
24 Export column number to environment, so it's available for retrieval. 
25 
26 
2: 
28 awkscript-'( total += S$ENVIRON["column number"] } 
29 END 14 prime Coral p 
30 Yes, a variable can hold an awk script. 
Sut 
92 
33 Now, run the awk script. 
34 awk "Sawkscript" "$filename" 
35) 
36 Thanks, Stephane Chazelas. 
37 
38 exit 0 


į ) It is possible to initialize and export variables in the same operation, as in export 
varl-xxx. 


However, as Greg Keraunen points out, in certain situations this may have a different 
effect than setting a variable, then exporting it. 


bash$ export var=(a b); echo $(var[0]) 
(a b) 


bash$ var=(a b); export var; echo ${var[0]} 
a 


declare, typeset 
The declare and typeset commands specify and/or restrict properties of variables. 
readonly 
Same as declare -r, sets a variable as read-only, or, in effect, as a constant. Attempts to change the 
variable fail with an error message. This is the shell analog of the C language const type qualifier. 
getopts 
This powerful tool parses command-line arguments passed to the script. This is the Bash analog of the 
getopt external command and the getopt library function familiar to C programmers. It permits 
passing and concatenating multiple options [5] and associated arguments to a script (for example 


scriptname -abc -e /usr/local). 


The getopts construct uses two implicit variables. SOPTIND is the argument pointer (OPTion INDex) 
and SOPTARG (OPTion ARGument) the (optional) argument attached to an option. A colon following 
the option name in the declaration tags that option as having an associated argument. 


A getopts construct usually comes packaged in a while loop, which processes the options and 
arguments one at a time, then increments the implicit SOPTIND variable to point to the next. 


1. The arguments passed from the command-line to the script must be preceded 
by a dash (-). It is the prefixed - that lets getopts recognize command-line 
arguments as options. In fact, getopts will not process arguments without the 
prefixed -, and will terminate option processing at the first argument 
encountered lacking them. 

2. The getopts template differs slightly from the standard while loop, in that it 
lacks condition brackets. 

3. The getopts construct is a highly functional replacement for the traditional 
getopt external command. 


1 while getopts ":abcde:fg" Option 
Z x» Umaienell ceclearer om, 
3 # a, b, c, d, e, f£, and g are the options (flags) expected. 
4 4$ The : after option 'e' shows it will have an argument passed with it. 
5 do 
6 case SOption in 
7 a) # Do something with variable 'a'. 
8 b) # Do something with variable 'b'. 
9 Sane 
10 e) 4 Do something with 'e', and also with SOPTARG, 
L1 # which is the associated argument passed with option 'e'. 
1,2 areas 
3 g ) # Do something with variable 'g'. 
4 esac 
15 done 
1G shire g(euommIEMJD = 1)) 
7 # Move argument pointer to next. 
8 
9 # All this is not nearly as complicated as it looks «grin». 


Example 15-21. Using getopts to read the options/arguments passed to a script 


1 #!/bin/bash 
2 # ex33.sh: Exercising getopts and OPTIND 
3 4 Script modified 10/09/03 at the suggestion of Bill Gradwohl. 
4 
E 
6 Here we observe how 'getopts' processes command-line arguments to script. 
Ji The arguments are parsed as "options" (flags) and associated arguments. 
8 
9 Tey inyoking taila Serie wras 
10 'scriptname -mn' 
ital 'scriptname -oq qOption' (qOption can be some arbitrary string.) 
1.2 'scriptname -qXXX ri 
1.3 
14 'scriptname -gr' 
15 #+ Unexpected result, takes "r" as the argument to option "q" 


16 'scriptname -q -r' 


1 
18 
19 
20 
21 
22 
23 
24 
25 
26 
2] 
28 
29 
30 
31 
32 
33 
34 
39 
36 
37 
38 
3 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
5L 
52 
53 
54 
55 
56 
5:3 
58 
3$ 
60 
61 
62 


t Unexpected result, same as above 
'scriptname -mnop -mnop' Unexpected result 
(OPTIND is unreliable at stating where an option came from.) 


If an option expects an argument ("flag:"), then it will grab 
+ whatever is next on the command-line. 


NO. ARGS-0 
E OPTERROR-85 


if [ $4 -eq "SNO ARGS" ] # Script invoked with no command-line args? 
then 
echo "Usage: 'basename $0' options (-mnopqrs)" 
exit $E OPTERROR # Exit and explain usage. 
# Usage: scriptname -options 
# Note: dash (-) necessary 


ital 


while getopts ":mnopq:rs" Option 


do 
case SOption in 
m ) echo "Scenario #1: option -m- [OPTIND-$[(OPTIND]]";; 
m | © ) emo VSgsie9 528 Gor oi -—SOnbiEdgim- LOI LIND=S ORTIND IN 
p ) echo "Scenario 43: option -p- IL Gier ID S {OL OIN | B e 
q ) echo "Sicemeiene G49 GoELON -GFN 
with argument \"SOPTARG\" [OPTIND-$[(OPTIND]]";; 
# Note that option 'q' must have an associated argument, 
#+ otherwise it falls through to the default. 
iz | e J echo Ves 758 jm —S9gidigim-/p» n 
* ) echo "Unimplemented option chosen.";; # Default. 
esac 
done 
shire S((SOLTIND = 3j 


#  Decrements the argument pointer so it points to next argument. 
# $1 now references the first non-option item supplied on the command-line 
#+ if one exists. 


exit $? 


# As Bill Gradwohl states, 

# "The getopts mechanism allows one to specify:  scriptname -mnop -mnop 
#+ but there is no reliable way to differentiate what came 

#+ from where by using OPTIND." 

# There are, however, workarounds. 


Script Behavior 


source, . (dot command) 
This command, when invoked from the command-line, executes a script. Within a script, a source 
file-name loads the file £11e-name. Sourcing a file (dot-command) imports code into the script, 
appending to the script (same effect as the #include directive in a C program). The net result is the 
same as if the "sourced" lines of code were physically present in the body of the script. This is useful 
in situations when multiple scripts use a common data file or function library. 


Example 15-22. "Including" a data file 


JL 
2 


#!/bin/bash 


data-file # Load a data file. 
# Same effect as "source data-file", but more portable. 


# The file "data-file" must be present in current working directory, 
#+ since it is referred to by its 'basename'. 


echo "variablel (from data-file) $variablel" 
echo "variable3 (from data-file) = $variable3" 


L2 
i 
let "sum = $variable2 + $variable4" 

echo "Sum of variable2 + variable4 (from data-file) = $sum" 
echo "messagel (from data-file) is \"Smessagel1\"" 

# Note: escaped quotes 


3 
4 
5 
6 
E 
8 
9 # Now, reference some data from that file. 
0 
i 
3 
4 
5 
6 
y 


iL 


18 

19 print message This is the message-print function in the data-file. 
20 

21 

22 exit 0 


File data-£ile for Example 15-22, above. Must be present in same directory. 


# This is a data file loaded by a script. 
Files of this type may contain variables, functions, etc. 
# It may be loaded with a 'source' or '.' command by a shell script. 


+ 


# Let's initialize some variables. 


variablel-22 
variable2-474 
variable3-5 
variable4-97 


messagel-"Hello, how are you?" 
message2-"Enough for now. Goodbye." 


e 


epo! Ge 
OY O' 4& Q). I9. IB. O (o O0 -1 O Or iS 0) IN S 


print message () 
{ 


# Echoes any message passed to it. 


de | —- VSL" | 
then 

return 1 

# Error, if argument missing. 
ftu 


Nere 
€x O oO s 


N N 
N e 


No NO PO 
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echo 


No PN 
=| O0 


(ies dp oz UI q 
do 
# Step through arguments passed to function. 
exeo =m US 
# Echo args one at a time, suppressing line feeds. 
cecho mn WW 
# Insert spaces between words. 
SANER 
# Next one. 
done 


N 
œ 


Ww CO D 
AIGO 


[99 
N 


echo 


CO0 CO CO CO CO CO CO 
$0 0 -—-10) 01 BW 


[i 
e) 


return 0 
41 } 


exit 


If the sourced file is itself an executable script, then it will run, then return control to the script that 
called it. A sourced executable script may use a return for this purpose. 


Arguments may be (optionally) passed to the sourced file as positional parameters. 


1 source $filename $argl arg2 
It is even possible for a script to source itself, though this does not seem to have any practical 
applications. 


Example 15-23. A (useless) script that sources itself 


1 #!/bin/bash 
2 4 self-source.sh: a script sourcing itself "recursively." 
3 sb prom Vrud Seit 1»23xeke," Wolltme 11. 
4 
5 MAXPASSCNT-100 # Maximum number of execution passes. 
6 
T7 echo =m VSpass cow Y 
8 At first execution pass, this just echoes two blank spaces, 
9 #+ since $pass count still uninitialized. 
10 
JL3L Jhexe Voss cominie qa dU 
12 Assumes the uninitialized variable $pass count 
13 #+ can be incremented the first time around. 
14 This works with Bash and pdksh, but 
15 #+ it relies on non-portable (and possibly dangerous) behavior. 
16 Better would be to initialize $pass count to 0 before incrementing. 
db) 
18 while [ "$pass count" -le $MAXPASSCNT | 
Le clo 
20 $0 # Script "sources" itself, rather than calling itself. 
ut # ./$0 (which would be true recursion) doesn't work here. Why? 
22 done 
23 
24 What occurs here is not actually recursion, 
25 #+ since the script effectively "expands" itself, i.e., 
26 #+ generates a new section of code 
27 #+ with each pass through the 'while' loop', 
28 with each 'source' in line 20. 
29 
30 Of course, the script interprets each newly 'sourced' "4!" line 
31 #+ as a comment, and not as the start of a new script. 
32 
33 echo 
34 
35 Grae () # The net effect is counting from 1 to 100. 
36 # Very impressive. 
aT 
38 # Exercise: 
Sg du) cessas 


40 # Write a script that uses this trick to actually do something useful. 


Unconditionally terminates a script. [6] The exit command may optionally take an integer argument, 
which is returned to the shell as the exit status of the script. It is good practice to end all but the 
simplest scripts with an exit 0, indicating a successful run. 


` If a script terminates with an exit lacking an argument, the exit status of the script is 
the exit status of the last command executed in the script, not counting the exit. This is 


exec 


shopt 


equivalent to an exit $?. 


$^) An exit command may also be used to terminate a subshell. 


This shell builtin replaces the current process with a specified command. Normally, when the shell 
encounters a command, it forks off a child process to actually execute the command. Using the exec 
builtin, the shell does not fork, and the command exec'ed replaces the shell. When used in a script, 
therefore, it forces an exit from the script when the exec'ed command terminates. [7] 


Example 15-24. Effects of exec 


(Eh 
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#!/bin/bash 

GEC echo MEn NUN oU) # Exit from script here. 

# 

# The following lines never execut 

echo "This echo will never echo." 

exit 99 # This script will not exit here. 
# Check exit value after script terminates 
#+ with an 'echo $?'. 
p dit wngDl uxo be 99. 


Example 15-25. A script that exec's itself 


NOB pop pppHppÀDGÀG:! 
O (o 0» -1 O Ui i& (). M HIS. O xo 0 -1 O O1 i Q Io E 


DRPD 
w N e 


#!/bin/bash 
# self-exec.sh 


# Note: Set permissions on this script to 555 or 755, 
# then call it with ./self-exec.sh or sh ./self-exec.sh. 


echo 


echo "This line appears ONCE in the script, yet it keeps echoing." 
echo Vine PID Cie this instance OE the seripët is still S9," 


# Demonstrates that a subshell is not forked off. 

chg “ haiie CUl- to ereit Y 
sleep 1 
exec $0 # Spawns another instance of this same script 


#+ that replaces the previous one. 
echo "This line will never echo!" # Why not? 


eui 99 # Will not exit here! 
# Exit code will not be 99! 


An exec also serves to reassign file descriptors. For example, exec «zzz-file replaces st din 
with the file zzz-£ile. 


$^) The -exec option to find is not the same as the exec shell builtin. 


caller 


This command permits changing shell options on the fly (see Example 25-1 and Example 25-2). It 
often appears in the Bash startup files, but also has its uses in scripts. Needs version 2 or later of Bash. 


(ex (Gil dex (63 fey [5 


shopt -s cdspell 


# Allows minor misspelling of directory names with 'cd' 


cd /hpme 4 Oops! Mistyped '/home'. 


pwd # /home 


# The shell corrected th 


misspelling. 


Putting a caller command inside a function echoes to st dout information about the caller of that 
function. 


|obombpmopmowmn 
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16 


#!/bin/bash 


INCE Leia (() 


{ 


# Inside functionl (). 


caller © # Tell me about it. 


functionl # Line 9 of script. 


9 main test.sh 


AKRAARAAA 


# 
# 
i A^A^A^^ 
# 


cales 0) # Has no effect becaus 


Line number that the function was called from. 
Invoked from "main" part of script. 
Name of calling script. 


le e WOE melce A 3EQUOICUE LEI c 


A caller command can also return caller information from a script sourced within another script. 
Analogous to a function, this is a "subroutine call." 


You may find this command useful in debugging. 


Commands 


true 


false 


A command that returns a successful (zero) exit status, but does nothing else. 


bash$ true 
bash$ echo $? 


0 


# Endless loop 
while true $e culate) deus Ug 
do 

operation-1 

operation-2 


operation-n 


# Need a way to break out of loop or script will hang. 


done 


A command that returns an unsuccessful exit status, but does nothing else. 


bash$ 


false 


bash$ echo $? 


i 


1 # Testing "false" 

2 if false 

3 then 

4 echo "false evaluates \"true\"" 
5 else 

6 echo "false evaluates \"false\"" 
T aa 

8 # false evaluates "false" 

9 
10 
11 # Looping while "false" (null loop) 
12 while false 

ILS. lo 

14 # The following code will not execute. 
LS operation-1 

16 operation-2 

17 ume 

18 operation-n 

19 # Nothing happens! 
20 done 

type [cmd] 


Similar to the which external command, type cmd identifies "cmd." Unlike which, type is a Bash 
builtin. The useful —a option to type identifies keywords and builtins, and also locates system 
commands with identical names. 


bash$ type '[' 
los mo ohe orai Leim 
bash$ type -a '[' 
[35er a shs ll Toy Iie shin 
| xm weis i 


bash$ type type 
type is a shell builtin 


The type command can be useful for testing whether a certain command exists. 

hash [cmds] 
Records the path name of specified commands -- in the shell hash table [8] -- so the shell or script 
will not need to search the $PATH on subsequent calls to those commands. When hash is called with 
no arguments, it simply lists the commands that have been hashed. The -r option resets the hash 
table. 

bind 
The bind builtin displays or modifies readline [9] key bindings. 

help 
Gets a short usage summary of a shell builtin. This is the counterpart to whatis, but for builtins. The 
display of help information got a much-needed update in the version 4 release of Bash. 


bash$ help exit 

GrH ce [Da 
Exit the shell with a status of N. If N is omitted, the exit status 
is that of the last command executed. 


15.1. Job Control Commands 


Certain of the following job control commands take a job identifier as an argument. See the table at end of the 
chapter. 


jobs 
Lists the jobs running in the background, giving the job number. Not as useful as ps. 
$^, It is all too easy to confuse jobs and processes. Certain builtins, such as kill, disown, 
and wait accept either a job number or a process number as an argument. The fg, bg 
and jobs commands accept only a job number. 
bash$ sleep 100 & 
[i] 1984 
bash $ jobs 
[1]* Running sleep 100 & 
"]" is the job number (jobs are maintained by the current shell). "1384" is the PID or 
process ID number (processes are maintained by the system). To kill this job/process, 
either a kill %1 or a kill 1384 works. 
Thanks, S.C. 
disown 
Remove job(s) from the shell's table of active jobs. 
fg. bg 
The fg command switches a job running in the background into the foreground. The bg command 
restarts a suspended job, and runs it in the background. If no job number is specified, then the fg or bg 
command acts upon the currently running job. 
wait 


Suspend script execution until all jobs running in background have terminated, or until the job number 
or process ID specified as an option terminates. Returns the exit status of waited-for command. 


You may use the wait command to prevent a script from exiting before a background job finishes 
executing (this would create a dreaded orphan process). 


Example 15-26. Waiting for a process to finish before proceeding 


!/bin/bash 


ROOT_UID=0 # Only users with SUID 0 have root privileges. 
E_NOTROOT=65 
E_NOPARAMS=66 


xar [jp VSI —- TSROOT OND T 

then 
echo "Must be root to run this script." 
# "Run along kid, it's past your bedtime." 
exit $E NOTROOT 


LE p ppppppmDot 
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if [ -z woz" ] 


echo "Usage: 'basename $0° find-string" 
exit $E NOPARAMS 


20 

21 echo "Updating 'locate' database..." 

22 echo "This may take a while." 

23 updatedb /usr & # Must be run as root. 

24 

25 wait 

26 # Don't run the rest of the script until 'updatedb' finished. 
27 4 You want the the database updated before looking up the file name. 
28 

29 locate $1 

30) 

31 # Without the 'wait' command, in the worse case scenario, 

32 #+ the script would exit while 'updatedb' was still running, 
33 #+ leaving it as an orphan process. 

34 

35 exit 0 


Optionally, wait can take a job identifier as an argument, for example, wait%1or wait SPPID. 
[10] See the job id table. 


į ) Within a script, running a command in the background with an ampersand (&) may 
cause the script to hang until ENTER is hit. This seems to occur with commands that 
write to stdout. It can be a major annoyance. 


1 #!/bin/bash 
2 4 test.sh 

3 

A Jg ol & 

5 echo "Done." 


bash$ ./test.sh 


Done. 
[bozo@localhost test-scripts]$ total 1 
IAW RE 1 bozo bozo 34 Oer ii 15709 test sa 


As Walter Brameld IV explains it: 


As far as I can tell, such scripts don't actually hang. It just 

seems that they do because the background command writes text to 
the console after the prompt. The user gets the impression that 

the prompt was never displayed. Here's the sequence of events: 


1. Script launches background command. 

2. Script exits. 

3. Shell displays the prompt. 

4. Background command continues running and writing text to the 
console. 

5. Background command finishes. 

6. User doesn't see a prompt at the bottom of the output, thinks script 
is hanging. 


Placing a wait after the background command seems to remedy this. 


#!/bin/bash 
# test.sh 


E O D = 


lg -l & 


5 echo "Done." 


6 wait 
bash$ ./test.sh 
Done. 
[bozo@localhost test-scripts]$ total 1 
—IEWESIE XR X 1 bozo bozo 34 (xeu iil 15309 testos 


Redirecting the output of the command to a file or even to /dev/nu11 also takes 
care of this problem. 
suspend 
This has a similar effect to Control-Z, but it suspends the shell (the shell's parent process should 
resume it at an appropriate time). 
logout 
Exit a login shell, optionally specifying an exit status. 
times 
Gives statistics on the system time elapsed when executing commands, in the following form: 


0m0.020s 0m0.020s 
This capability is of relatively limited value, since it is not common to profile and benchmark shell 


scripts. 
kill 
Forcibly terminate a process by sending it an appropriate terminate signal (see Example 17-6). 
Example 15-27. A script that kills itself 
1 #!/bin/bash 
2 # self-destruct.sh 
3 
4 kill $$ # Script kills its own process here. 
5 i? xeu tici VSS me rns fenes y PID), 
6 
7 echo "This line will not echo." 
8 # Instead, the shell sends a "Terminated" message to stdout. 
9 
10 exit 0 # Normal exit? No! 
11 
1,2 After this script terminates prematurely, 
13 #+ what exit status does it return? 
14 
LS Sl, gmiedt-eles eue Sn 
16 exco P 
iy $ 143 
18 
19 AS = LAS ap 305 
20 TERM signal 
HA kill -1 lists all the signals (as does the file /usr/include/asm/signal.h). 
A kill -9 isa sure kill, which will usually terminate a process that stubbornly 
refuses to die with a plain kill. Sometimes, a kil11 -15 works. A zombie process, 
that is, a child process that has terminated, but that the parent process has not (yet) 
killed, cannot be killed by a logged-on user -- you can't kill something that is already 
dead -- but init will generally clean it up sooner or later. 
killall 


The killall command kills a running process by name, rather than by process ID. If there are multiple 
instances of a particular command running, then doing a killall on that command will terminate them 
all. 


$^) This refers to the killall command in /usr/bin, not the killall script in 
fete/re.d/init.d. 
command 
The command directive disables aliases and functions for the command immediately following it. 


bash$ command 1s 


<=) This is one of three shell directives that effect script command processing. The others 
are builtin and enable. 
builtin 
Invoking builtin BUILTIN COMMAND runs the command BUTLTIN COMMAND as a shell 
builtin, temporarily disabling both functions and external system commands with the same name. 
enable 
This either enables or disables a shell builtin command. As an example, enable -n kill disables 
the shell builtin kill, so that when Bash subsequently encounters kill, it invokes the external command 
/bin/kill. 


The -a option to enable lists all the shell builtins, indicating whether or not they are enabled. The -f 
filename option lets enable load a builtin as a shared library (DLL) module from a properly 
compiled object file. [11]. 

autoload 
This is a port to Bash of the ksh autoloader. With autoload in place, a function with an autoload 
declaration will load from an external file at its first invocation. [12] This saves system resources. 


Note that autoload is not a part of the core Bash installation. It needs to be loaded in with enable 
-f (see above). 


Table 15-1. Job identifiers 


T 


"current" job (last job stopped in foreground or started in background) 
Last job 


Last background process 


X 


fe) 


oe 


oo 
3 [Qo 


+ 


= |i 


% 
% 
E 
% 


Notes 


[1] As Nathan Coulter points out, "while forking a process is a low-cost operation, executing a new 
program in the newly-forked child process adds more overhead." 


[2] Anexception to this is the time command, listed in the official Bash documentation as a keyword 
("reserved word"). 

[3] Note that /et cannot be used for setting string variables. 

[4] To Export information is to make it available in a more general context. See also scope. 

I5] 


An option is an argument that acts as a flag, switching script behaviors on or off. The argument 
associated with a particular option indicates the behavior that the option (flag) switches on or off. 


[6] Technically, an exit only terminates the process (or shell) in which it is running, not the parent process. 
[7] Unless the exec is used to reassign file descriptors. 
[8] 


Hashing is a method of creating lookup keys for data stored in a table. The data items themselves are 
"scrambled" to create keys, using one of a number of simple mathematical algorithms (methods, or 


recipes). 


An advantage of hashing is that it is fast. A disadvantage is that collisions -- where a single key maps to 
more than one data item -- are possible. 


For examples of hashing see Example A-20 and Example A-21. 
[9] The readline library is what Bash uses for reading input in an interactive shell. 
[10] This only applies to child processes, of course. 


[11] The C source for a number of loadable builtins is typically found in the 
/usr/share/doc/bash-?.??/functions directory. 


Note that the -£f option to enable is not portable to all systems. 
[12] The same effect as autoload can be achieved with typeset -fu. 
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Chapter 16. External Filters, Programs and 
Commands 


Standard UNIX commands make shell scripts more versatile. The power of scripts comes from coupling 
system commands and shell directives with simple programming constructs. 


16.1. Basic Commands 


The first commands a novice learns 


Is 


The basic file "list" command. It is all too easy to underestimate the power of this humble command. 


For example, using the —R, recursive option, Is provides a tree-like listing of a directory structure. 
Other useful options are —S, sort listing by file size, ^t, sort by file modification time, -b, show 
escape characters, and —i, show file inodes (see Example 16-4). 


The /s command returns a non-zero exit status when attempting to list a non-existent 
file. 


bash$ ls abc 
ls: abc: No such file or directory 


bash$ echo $? 
2 


Example 16-1. Using /s to create a table of contents for burning a CDR disk 


NOR pop pop ppp: 
O (o 0» -1 O Ui i (). M. I. O xo 0 -1 O O1 i QI E 


NO NO BO PO NO PO ND 
sup cx, (ar de» Ge) [er qp 


N 
[99] 


C9 CO CO CO CO CO CO CO CO Dd 
Oo =o, OIE ® bee © 


#!/bin/bash 
# ex40.sh (burn-cd.sh) 
# Script to automate burning a CDR. 


SPEED-10 # May use higher speed if your hardware supports it. 
IMAGEFILE-cdimage.iso 

CONTENTSFILE-contents 

# DEVICE-/dev/cdrom For older versions of cdrecord 

DEVICE-"1,0,0" 

DEFAULTDIR-/opt # This is the directory containing the data to be burned. 


# Make sure it exists. 
# Exercise: Add a test for this. 


# Uses Joerg Schilling's "cdrecord" package: 
# http://www.fokus.fhg.de/usr/schilling/cdrecord.html 


# If this script invoked as an ordinary user, may need to suid cdrecord 
#+ chmod uts /usr/bin/cdrecord, as root. 
# Of course, this creates a security hole, though a relatively minor one. 


iE [ =7 USA ] 
then 
IMAGE DIRECTORY-S$DEFAULTDIR 


# Default directory, if not specified on command-line. 
else 


IMAGE DIRECTORY-$1 


EIL 


Create a "table of contents" file. 

ls -1RF SIMAGE DIRECTORY > $IMAGE DIRECTORY/S$CONTENTSFILFE 
The "1" option gives a "long" file listing. 

The "R" option makes the listing recursive. 

The "F" option marks the file types (directories get a trailing /). 
echo "Creating table of contents." 


Create an image file preparatory to burning it onto the CDR. 
mkisofs -r -o SIMAGEFILE SIMAGE DIRECTORY 


cat, tac 


rev 


cp 


99 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 


echo "Creating ISO9660 file system image (SIMAGEFILE) ." 


# Burn the CDR. 
echo "Burning the disk." 
echo "Please be patient, this will take a while." 


wodim 


v -isosize dev=SDEVICE SIMAGEFILE 


# In newer Linux distros, the "wodim" utility assumes the 
#+ functionality of "cdrecord." 
exitcode-$? 


echo 


Exit code = Sexitcode" 


Xit Sexitcod 


cat, an acronym for concatenate, lists a file to stdout. When combined with redirection (> or >>), it 
is commonly used to concatenate files. 


1 
2 
3 
4 


# Uses of 'cat' 
cat filename p Wiers che Cile, 


car silenl stale 2 ES S 2 alas IAS # Combines three files into one. 


The -n option to cat inserts consecutive numbers before all lines of the target file(s). The -b option 
numbers only the non-blank lines. The -v option echoes nonprintable characters, using ^ notation. 
The -s option squeezes multiple consecutive blank lines into a single blank line. 


See also Example 16-28 and Example 16-24. 


$^, In a pipe, it may be more efficient to redirect the st din to a file, rather than to cat the 
file. 


JL ear inlemenme || ice mem ATZ 


2 
3 tr a-z A-Z « filename # Same effect, but starts one less process, 
4 #+ and also dispenses with the pipe. 


tac, is the inverse of cat, listing a file backwards from its end. 


reverses each line of a file, and outputs to st dout. This does not have the same effect as tac, as it 
preserves the order of the lines, but flips each one around (mirror image). 


bash$ cat filel.txt 
Tass sls lee i. 
dae se dae Z, 


bash$ tac filel.txt 
ais sig dame Zo 
rie: als lire i. 


bash$ rev filel.txt 
sil (ueollL Sal augur 
o2 Sinai, Sal sala 


This is the file copy command. cp filel file2 copies filel to file2, overwriting £ile2 if 
it already exists (see Example 16-6). 


Particularly useful are the —a archive flag (for copying an entire directory tree), the 
-u update flag (which prevents overwriting identically-named newer files), and the 


-r and -R recursive flags. 


il cp -u egouueem obue/** dest ql: 
2 # "Synchronize" dest dir to source dir 
3 #+ by copying over all newer and not previously existing files. 


This is the file move command. It is equivalent to a combination of cp and rm. It may be used to 
move multiple files to a directory, or even to rename a directory. For some examples of using mv in a 


script, see Example 10-11 and Example A-2. 


cg; When used in a non-interactive script, mv takes the -f (force) option to bypass user 
input. 


When a directory is moved to a preexisting directory, it becomes a subdirectory of the 
destination directory. 


bash$ mv source directory target directory 
bash$ ls -1F target directory 


total d 
drwXrWXr-x 2 bozo bozo 1024 May 28 19:20 source_directory/ 


rm 


Delete (remove) a file or files. The -f option forces removal of even readonly files, and is useful for 
bypassing user input in a script. 


B 
The rm command will, by itself, fail to remove filenames beginning with a dash. 
Why? Because rm sees a dash-prefixed filename as an option. 


bash$ rm -badname 
ams 3g oprion == Ig 
Try rm --help' for more information. 
One clever workaround is to precede the filename with a" -- " (the end-of-options 


flag). 


bash$ rm -- -badname 
Another method to is to preface the filename to be removed with a dot-slash. 


bash$ rm ./-badname 


When used with the recursive flag —r, this command removes files all the way down 
the directory tree from the current directory. A careless rm -rf * can wipe out a big 
chunk of a directory structure. 
rmdir 
Remove directory. The directory must be empty of all files -- including "invisible" dotfiles [1] -- for 
this command to succeed. 
mkdir 
Make directory, creates a new directory. For example, mkdir -p 
project/programs/December creates the named directory. The -p option automatically 
creates any necessary parent directories. 
chmod 
Changes the attributes of an existing file or directory (see Example 15-14). 


1 chmod +x filename 
2 4 Makes "filename" executable for all users. 


chmod u*s filename 

Sets "suid" bit on "filename" permissions. 

An ordinary user may execute "filename" with same privileges as the file's owner. 
(This does not apply to shell scripts.) 


sup Oy (Gn) dex GS) 


chmod 644 filename 
Makes "filename" readable/writable to owner, readable to others 
(octal mode). 


chmod 444 filename 
Makes "filename" read-only for all. 
Modifying the file (for example, with a text editor) 
+ not allowed for a user who does not own the file (except for root), 
+ and even the file owner must force a file-save 
a ase ghe mochiries Ere Pile, 
Same restrictions apply for deleting the file. 


[= G&S) Wey (Go | en a) des Ws) Iss) I 


PR 


chmod 1777 directory-name 
Gives everyone read, write, and execute permission in directory, 
+ however also sets the "sticky bit". 
This means that only the owner of the directory, 
+ owner of the file, and, of course, root 
+ can delete any particular file in that directory. 


chmod 111 directory-name 
Gives everyon xecute-only permission in a directory. 
This means that you can execute and READ the files in that directory 
* (execute permission necessarily includes read permission 
* because you can't execute a file without being able to read it). 
But you can't list the files or search for them with the "find" command. 
These restrictions do not apply to root. 


PPP PE 
YADUBRWNPFPOOMIDUBRWNHE 


chmod 000 directory-name 
No permissions at all for that directory. 
Cante reac, write, (ue GXSCUES Piles aia ic. 
(eum ib even oet tiles aia at or Ye to XE. 
But, you can rename (mv) the directory 
+ or delete it (rmdir) if it is empty. 
You can even symlink to files in the directory, 
but you can't read, write, or execute the symlinks. 
These restrictions do not apply to root. 


m oH 


B oag 
C Xo oo 


N N 
N E 


N 
[99 
1 
t 


N 
A 


chattr 
Change file attributes. This is analogous to chmod above, but with different options and a different 
invocation syntax, and it works only on ext2/ext3 filesystems. 


One particularly interesting chattr option is i. A chattr +i filename marks the file as immutable. 
The file cannot be modified, linked to, or deleted, not even by root. This file attribute can be set or 


removed only by root. In a similar fashion, the a option marks the file as append only. 


root# chattr +i filel.txt 


root# rm filel.txt 


rm: remove write-protected regular file `filel.txt'? y 
rm: cannot remove `filel.txt': Operation not permitted 


If a file has the s (secure) attribute set, then when it is deleted its block is overwritten with binary 
zeroes. [2] 


In 


If a file has the u (undelete) attribute set, then when it is deleted, its contents can still be retrieved 
(undeleted). 


If a file has the c (compress) attribute set, then it will automatically be compressed on writes to disk, 
and uncompressed on reads. 


$^) The file attributes set with chattr do not show in a file listing (Is -L). 


Creates links to pre-existings files. A "link" is a reference to a file, an alternate name for it. The In 
command permits referencing the linked file by more than one name and is a superior alternative to 
aliasing (see Example 4-6). 


The In creates only a reference, a pointer to the file only a few bytes in size. 


The In command is most often used with the —s, symbolic or "soft" link flag. Advantages of using the 
—s flag are that it permits linking across file systems or to directories. 


The syntax of the command is a bit tricky. For example: ln -s oldfile newfile links the 
previously existing oldfile to the newly created link, newfile. 


D If a file named newfile has previously existed, an error message will result. 


Which type of link to use? 


As John Macdonald explains it: 


Both of these [types of links] provide a certain measure of dual reference -- if you edit the contents 


of the file using any name, your changes will affect both the original name and either a hard or soft 
new name. The differences between them occurs when you work at a higher level. The advantage of 
a hard link is that the new name is totally independent of the old name -- if you remove or rename 
the old name, that does not affect the hard link, which continues to point to the data while it would 
leave a soft link hanging pointing to the old name which is no longer there. The advantage of a soft 
link is that it can refer to a different file system (since it is just a reference to a file name, not to 
actual data). And, unlike a hard link, a symbolic link can refer to a directory. 


Links give the ability to invoke a script (or any other type of executable) with multiple names, and 
having that script behave according to how it was invoked. 


Example 16-2. Hello or Good-bye 


1 #!/bin/bash 

2 hello.sh: Saying "hello" or "goodbye" 

3 #+ depending on how script is invoked. 
4 

5 Make a link in current working directory ($PWD) to this script: 
6 ln -s hello.sh goodbye 

7 Now, try invoking this script both ways: 

8 ./hello.sh 

9 ./goodbye 
10 
LL 
12 HELLO_CALL=65 


13 GOODBYE CALL-66 


14 

1S muc | S = "serie | 

16 then 

L7 echo "Good-bye!" 

18 # Some other goodbye-type commands, as appropriate. 
IY) exit $GOODBYE CALL 

2O itx 

Zl 


22 echo Hello 
23 # Some other hello-type commands, as appropriate. 
24 exit SHELLO CALL 


man, info 
These commands access the manual and information pages on system commands and installed 
utilities. When available, the info pages usually contain more detailed descriptions than do the man 


pages. 


There have been various attempts at "automating" the writing of man pages. For a script that makes a 
tentative first step in that direction, see Example A-39. 


Notes 


ul 


Dotfiles are files whose names begin with a dot, such as ~/ .Xdefaults. Such filenames do not 
appear in a normal Is listing (although an Is -a will show them), and they cannot be deleted by an 
accidental rm -rf *. Dotfiles are generally used as setup and configuration files in a user's home 
directory. 

[2] This particular feature may not yet be implemented in the version of the ext2/ext3 filesystem installed 
on your system. Check the documentation for your Linux distro. 
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16.2. Complex Commands 


Commands for more advanced users 
find 
-exec COMMAND \; 


Carries out COMMAND on each file that find matches. The command sequence terminates with ; (the 


;" is escaped to make certain the shell passes it to find literally, without interpreting it as a special 
character). 


bash$ find ~/ -name '*.txt' 
/home/bozo/.kde/share/apps/karm/karmdata.txt 
/home/bozo/misc/irmeyc.txt 
/home/bozo/test-scripts/1.txt 


If COMMAND contains {}, then find substitutes the full path name of the selected file for "{}". 


1 find ~/ -name 'core*' xec rm () V; 
2 Removes all core dump files from user's home directory. 
1 find /home/bozo/projects -mtime -1 
2 a Note minus sign! 
3 Lists all files in /home/bozo/projects directory tree 
4 #+ that were modified within the last day (current, day - 1). 
E 
6 find /home/bozo/projects -mtime 1 
p Same as above, but modified *exactly* one day ago. 
8 
9 mtime = last modification time of the target fil 
LO ctime = last status change time (via 'chmod' or otherwise) 
ial atime = last access time 
12 
13 DIR=/home/bozo/junk_files 
14 imel USIDIURV -rypa E -atine r5 Kee icim 4) We 
15 i dn 
16 Curly brackets are placeholder for the path name output by "find." 
Ly 
18 Deletes all files in "/home/bozo/junk files" 
19 #+ that have not been accessed in *at least* 5 days (plus sign ... +5). 
20 
2 "-type filetype", wher 
22 f = regular file 
23 d = directory 
24 1 = symbolic link, etc. 
25 
26 (The 'find' manpage and info page have complete option listings.) 


find /etc sexe Crees ~ [0-9] [0-9] [5] 10-9) [9-9] "1. ] 10=9] [O=9] "1. ] ro-91[[9-9]-*" 43 Ye 


Finds all IP addresses (xxx.xxx.xxx.xxx) in /etc directory files. 
There a few extraneous hits. Can they be filtered out? 


Possibly by: 


ioc) 4/eNE -type £ sie eee PY Ye || asse exe eheee ANa x 
spese "^q. ps9 ws od Ie No sls Hg a^ ol 2S? 


11 4$ [:digit:] is one of the character classes 
12 #+ introduced with the POSIX 1003.2 standard. 
i3 

14 # Thanks, Stéphane Chazelas. 


8"); The -exec option to find should not be confused with the exec shell builtin. 


Example 16-3. Badname, eliminate file names in current directory containing bad characters 
and whitespace. 


#!/bin/bash 
# badname.sh 
# Delete filenames in current directory containing bad characters. 


for filename in * 


do 
badname-'echo "$filename" | sed -n /[\+\{\7\"\\\=\2?~\ (N) NSNS2N&N*SNINST/p^ 
badname-'echo "$filename" | sed -m '/[*[;"N-?-()«»&*|$]/p'" also works. 
Deletes files containing these nasties: s { p "NooeTe()s&sgew!]s 


rm $badname 2>/dev/null 
^^^^^^^^^^^ Error messages deep-sixed. 


done 


Now, take care of files containing all manner of whitespace. 

find . -name "* *" —-exec rm -f {} V; 

The path name of the file that find finds replaces the "{}". 

The '\' ensures that the ';' is interpreted literally, as end of command. 


NPRPRPRPRP PPP PY 
O io 00 -1 O Oi i& (0. M. I. O xo 0 -1 O O1 iS CQ IO 5 


e» 0) 
2l 
22 
23 Commands below this line will not execute because of _exit_ command. 
24 
215) An alternative to the above script: 
26 find . -name '*[+{;"\\=?~()<>&*|S$ ]*' -maxdepth 0 \ 
27 -exec rm -f "{}' \; 
28 The "-maxdepth 0" option ensures that find will not search 
29 #+ subdirectories below $PWD. 
20 
cal (hanks, Oo...) 


Example 16-4. Deleting a file by its inode number 


!/bin/bash 
idelete.sh: Deleting a file by its inode number. 


This is useful when a filename starts with an illegal character, 
suchas Gu =, 


ARGCOUNT-1 # Filename arg must be passed to script. 
E WRONGARGS-70 

E FILE NOT EXIST-71 

E CHANGED. MIND-72 


if [ $4 -ne "SARGCOUNT" ] 

then 
echo "Usage: 'basename $0! filename" 
exit $E WRONGARGS 

ie aL 


|Oobomobpmopmpmowmn 
OY O1 4 (0. I9. IP. O xo 00 -1 O OI iS CQ) IN S 


17 

ig zi [ P e "ex" |] 

19 then 

20 exo Wille NUWSIUNM does mere exigit- 
Al exit Sa ILE, JEDXILSUE 


22 33 

23 

24. ius Jg =i | graa "SX" | aw: V fjowenime SIR" 

25 inum = inode (index node) number of file 

26 

27) Every file has an inode, a record that holds its physical address info. 
28 

29 

30 echo; echo -n "Are you absolutely sure you want to delete \"S$1\" (y/n)? " 
E Time "—w" (yoEom (io "ru" also aske tals. 

32 read answer 

33 case "$answer" in 

34 [nN]) echo "Changed your mind, huh?" 


38 exit $E CHANGED MIND 

36 P 

Sy =) echo Vineilereslioe, ee WUSgNU Wee 
38 esac 

39 

40 find . —inum $inum -exec rm () V; 

41 # (y 

42 # Curly brackets are placeholder 
43 #+ Or ESE (Quite low Viboa" 
44 echo "File "\"S1"\" deleted!" 

45 

46 exit 0 


The find command also works without the -exec option. 


#!/bin/bash 

# Find suid root files. 

# A strange suid file might indicate a security hole, 
#+ or even a system intrusion. 


directory-"/usr/sbin" 
#7 Wolginc also try /Sloim, Mola, /USe/olm, /msie/ilocalfoim, (MEG 
permissions="+4000" # suid root (dangerous!) 


(sy =a] fey) (yl des (63) [J 


Ko) 


11 for file in $( find "Sdirectory" -perm "Spermissions" ) 
12 Clo 

13 lg  —dipm ——guuchwou US le" 

14 done 


See Example 16-30, Example 3-4, and Example 11-9 for scripts using find. Its manpage provides 
more detail on this complex and powerful command. 

xargs 
A filter for feeding arguments to a command, and also a tool for assembling the commands 
themselves. It breaks a data stream into small enough chunks for filters and commands to process. 
Consider it as a powerful replacement for backquotes. In situations where command substitution fails 
with a too many arguments error, substituting xargs often works. [1] Normally, xargs reads from 
stdin or from a pipe, but it can also be given the output of a file. 


The default command for xargs is echo. This means that input piped to xargs may have linefeeds and 
other whitespace characters stripped out. 


bash$ 1s -1 
cOcal © 
=P ewe r=— 1 bozo bozo 0 dain 29 29256. stulte 


== iri iL bezo boza Q aem 29 295856 i192 


bash$ ls -1 | xargs 


total Q -—zruw-rw-i--— l bozo bozo 0 Jaa 29 25258 kilel =rw=-rw=-r=-= 1 bozo bozo 0 Jain... 


bash$ find ~/mail -type f | xargs grep "Linux" 

P miise User Agents esie AO OT sita) 

./sent-mail-jul-2005: hosted by the Linux Documentation Project. 
./sent-mail-jul-2005: (Linux Documentation Project Site, rtf version) 
./sent-mail-jul-2005: Subject: Criticism of Bozo's Windows/Linux article 
./sent-mail-jul-2005: while mentioning that the Linux ext2/ext3 filesystem 


ls | xargs -p -1 gzip gzips every file in current directory, one at a time, prompting before 
each operation. 


$^; Note that xargs processes the arguments passed to it sequentially, one at a time. 


bash$ find /usr/bin | xargs file 
/usr/bin: directory 
/usr/bin/foomatic-ppd-options: perl script text executabl 


An interesting xargs option is -n NN, which limits to NN the number of arguments 
passed. 


ls | xargs -n 8 echo lists the files in the current directory in 8 columns. 


į ) Another useful option is -0, in combination with find -print0Oorgrep -12. 
This allows handling arguments containing whitespace or quotes. 


find / -type f -printO | xargs -0 grep -liwZ GUI | xargs 
-0 rm -f 


grep -rliwZ GUI / | xargs -0 rm -f 
Either of the above will remove any file containing "GUI". (Thanks, S.C.) 
Or: 


cat /proc/"Spid"/"SOPTION" | xargs -0 echo 
# From Han Holl's fixup of "get-commandline.sh" 
Wr SCript im W/clew and foro! chapter, 


b w V a 


Example 16-5. Logfile: Using xargs to monitor system log 


#!/bin/bash 


# Generates a log file in current directory 
# from the tail end of /var/log/messages. 


de Gs mE» pL 


Note: /var/log/messages must be world readable 
if this script invoked by an ordinary user. 
#root chmod 644 /var/log/messages 


(INES-5 


BO p|ppbpmbpmpmpniurt 
O0 -1 OY Ui 4 CQ). I9. IB. O (0 00 -1 O HY 


( date; uname -a ) >>logfile 
Time and machine name 
cho >>logfile 
tail -n S$LINES /var/log/messages | xargs | fmt -s >>logfile 
echo >>logfile 
echo >>logfile 


19 exit 0 

20 

2 Note 

22 =se 

23 As Frank Wang points out, 

24 #+ unmatched quotes (either single or double quotes) in the source file 

25 #+ may give xargs indigestion. 

26 

2 He suggests the following substitution for line 15: 

28 tail -m SLUMS /var/log/ messages | tie -o "XN" | zarga | ime -s 109gtile 


# Exercise: 


# Modify this script to track changes in /var/log/messages at intervals 
#+ of 20 minutes. 
# Hint: Use the "watch" command. 


WWWW WW WW DN 
fom al dex (3 (SS) J Sy Wor 
+ 
| 
| 
| 
| 
| 
| 
| 
| 


As in find, a curly bracket pair serves as a placeholder for replacement text. 


Example 16-6. Copying files in current directory to another 


1 #!/bin/bash 

2 copydir.sh 

3 

4 Copy (verbose) all files in current directory ($PWD) 

5 #+ to directory specified on command-line. 

6 

7 E_NOARGS=85 

8 

Sabie [| c Wisk 7] # Exit if no argument given. 

10 then 

LL echo "Usage: `basename $0' directory-to-copy-to" 

12 exit $E NOARGS 

JS ses 

14 

15 ls a || zarge ~= © emp 1» Sil 

16 iym ue iu 

17 -t is "verbose" (output command-line to stderr) option. 

Le =1 is Wreplace siticlings! Cis LIS 

19 {} is a placeholder for output text. 
20 Tois Us Sime ie. che US OF a euel ocea josie aim Weisel. V 
21 
D ibgteWE dele. files aia Current directory (ils 3 
ZS) qr DEBS che ovcpve gt Vis” as arguments to "rugs" (- -E OPCS) 
24 #+ then copy (cp) these arguments ({}) to new directory ($1). 
25 


26 $ The net result is th xact equivalent of 

27 #+ ej * Sil 

28 #+ unless any of the filenames has embedded "whitespace" characters. 
29 

30 exit 0 


Example 16-7. Killing processes by name 


1 #!/bin/bash 

2 kill-byname.sh: Killing processes by name. 

3 Compare this script with kill-process.sh. 

4 

5 For instance, 

6 #+ try "./kill-byname.sh xterm" -- 

7 #+ and watch all the xterms on your desktop disappear. 

8 

9 Warning: 
d us Reeser 
ALT, Meus we ci reel cinereus seco c 
L2 Running it carelessly (especially as root) 
13 #+ can cause data loss and other undesirabl ffects. 
14 
15 E BADARGS-66 
16 
lg sit test - "SI se Ne Comman- line arg supplici 
18 then 

ig) echo "Usage: 'basename $0` Process(es) to kill" 
20 exit $E BADARGS 
Bil. stai 
22 
23 
24 PROCESS NAME-"$]1" 
25) pS ex || Cie VSPRNOCISS NS || awk prine SLY [| senes =i kill I} 2657s 
26 £i aS 
27 
28 
2 Notes: 
30 -i is the "replace strings" option to xargs. 
Sil The curly brackets are the placeholder for the replacement. 
32 2&>/dev/null suppresses unwanted error messages. 
319 
34 Can grep "SPROCESS NAME" be replaced by pidof "SPROCESS NAME"? 
35) 


ei eos. Se 


39 # The "killall" command has the same effect as this script, 
40 #+ but using it is not quite as educational. 


Example 16-8. Word frequency analysis using xargs 


! /bin/bash 
wf2.sh: Crude word frequency analysis on a text file. 


Uses 'xargs' to decompose lines of text into single words. 
Compare this example to the "wf.sh" script later on. 


(ger cp (ox (ap des Go MS) [E 


Check for input file on command-line. 


expr 


9 ARGCS=I 


10 E_BADARGS=85 

11 E_NOFILE=86 

12 

i3 ss || Su; -ma “GARGS” T 

14 Correct number of arguments passed to script? 
15 then 

ENG echo "Usage: 'basename $0! filename" 

17 exit $E BADARGS 

JG) i 

19 

29 age dp f exp Weg | # Check if file exists. 
21 then 


22 exeo Wrile NUSINU gloss io eis. 
29 exit $E NOFILE 


24 Fi 

25 

26 

2] 

28 FERRE EEE EE EEE HE HE EEE EE EH HE HE EEE EE EH HE EE EEE EE HEH HE 
29 cew "SI" || zargs sail | \ 

30 List the file, one word per line. 

31 tr A-Z a-z | \ 

32 Shift characters to lowercase. 

33 eed e e/a ee "s/N a —9 Vs J^ 

394. fg’ | \ 

35) Filter out periods and commas, and 

36 #+ change space between words to linefeed, 

37 eost Unig =e | sors =mi 

38 Finally remove duplicates, prefix occurrence count 
39 #+ and sort numerically. 

40 Ha HE TE FE FE HE E TE EE HE TE FE HE EERE EEE HE EE EEE EE E E E E E E EEE E E ERE 
41 

42 This does the same job as the "wf.sh" example, 

43 #+ but a bit more ponderously, and it runs more slowly (why?). 
44 

45 axit S? 


All-purpose expression evaluator: Concatenates and evaluates the arguments according to the 
operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, 
string, or logical. 


expr 3 + 5 
returns 8 
expr 5 % 3 
returns 2 
expr 1 / 0 
returns the error message, expr: division by zero 


Illegal arithmetic operations not allowed. 
expr 5 \* 3 
returns 15 


The multiplication operator must be escaped when used in an arithmetic expression with 
expr. 

y= expr Sy + 1° 
Increment a variable, with the same effect as let y=y+1 and y=$ (($y*1)). This is an 


example of arithmetic expansion. 
z=expr substr $string Sposition $length' 


Extract substring of Slength characters, starting at $position. 


Example 16-9. Using expr 


Mop p opp ppppGÀ:! 
O (o 0» -1 O Oi i (). MN PB. O xo 0 -1 O O1 4 QI E 


[NOP ISS) TROY TSS) RESO SO ME IS 
eu] Ge» Gal dE Go) de» (5 


BWWW C0 C0 C0 CO CO CO CO hM) Dd 
S) Wer (es) SI Gy (On des (es (RSS p m» we (oo 


[i 
[Eus 


Of 4 bb SP SP aum aum 
(S) Wey (oe) —] fex, (np dex. (eor [x5 


(On On (Ont GL. Gah Gah (On Gal ori 
Wo (os) c py (On ES (£5 sy (5 


60 


#!/bin/bash 


# Demonstrating some of the uses of 


'expr' 


# 


echo 


# Arithmetic Operators 


echo "Arithmetic Operators" 


echo 
a= expr 5 + 3° 
echo "5 4 9 = Saw 


GI Exgoie Sei ae il” 


echo 
acing: Ya + d = Sau 

echo "(incrementing a variable)" 
a= expr 5$ 3° 

# modulo 

echo 

ceno "5 Suse 3 = Sal 

echo 

echo 


# Logical Operators 
# 


n2 enema) dL. aie ETUS) 


() slit stallse, 


#+ opposite of normal Bash convention. 


echo "Logical Operators" 


echo 


x-24 
y=25 
b=expr $x = 
echo "b = $b" 
echo 


Sy^ 


a=3 
b- expr $a MV» 10° 


# Test 
# 0 ( 


echo 'b-'expr $a MV» 10°, therefo 
echo. "qup coy b= a Ease 
echo "b = Sb" $0 ( 
echo 

b-' expr $a \< 10° 

echo "TE a= 10, ba (true 
echo "b = Sb" sd. aY 
echo 

# Note escaping of operators. 
b-' expr $a \<= 3° 

echo "If a <= 3, b= 1 (true)" 
echo "b = Sb" sod d 


i There xe algo a Us" 


operator 


equality. 
$x -ne Sy ) 


Mte 


Sh ene dili. y 


3 =e 3 ) 
(greater than or equal 


to). 


61 

62 

63 echo 

64 echo 

65 

66 

67 

68 4 String Operators 
69 # 
70 
71 echo "String Operators" 

72 echo 

73 

74 a=1234zipper43231 

VS acho Ville string being ojoeuealeecl woon ia CY SDN. UU 
76 

77 # length: length of string 

78 b= expr length $a” 

US) echo Vineineiela ene \VSa\Y she Slo. U 


80 
Sil Gp mces posltlon OI kiret Character im Silosiciciing 
82 # that matches a character in string 


83 b=`expr index $a 23^ 

84 ocho VWNuINe Les posicion wie first C WU am WUSBNU im Eou" 

85 

86 4$ substr: extract substring, starting position & length specified 
87 b= expr substr $a 2 6° 

88 ocho "SulosiExiuas gu NUSENU. starting ar [SoSuiExGm 2, \ 

G9 and 6 chare long as ebt.” 

90 

om 

92 # The default behavior of the 'match' operations is to 

93 #+ search for the specified match at the BEGINNING of the string. 
94 # 


DINE Using Regular Expressions 
MG lo eroe maten "Se Ugnoojpwu # Numerical count. 
97 echo Number of digits at the beginning of \"Sa\" is Sb. 
96 b= expr match Val VN (1OS91 NS # Note that escaped parentheses 
99 # == == #+ trigger substring match. 
100 sche “the chiens ei tho lepus of Yat eue eoon 
101 
102 echo 
103 
104 exit 0 


(D The : (null) operator can substitute for match. For example, b=" expr $a : [0-9]*" is the 
exact equivalent of bz? expr match $a [0-9]*'" in the above listing. 


#!/bin/bash 


echo 

echo YString operations usine \Yexgore \Ssicwiing 2 WV construct" 
cho " " 
echo 


a=1234zipper5FLIPPER43231 


echo tmherstring being operated upon as WW erpr USE =e UX(f SN) Yee 
Escaped parentheses grouping operator. == eie 


# KKKKKKKKKKKKKKKKKKKKKKKKKKK 


#+ Escaped parentheses 


#+ match a substring 
# ck ck ck ck ck Ck ck ck ck ck ck ck kk kk kk KKK ko ko ko 


|ÉBnbmbbmmpwuinLun 
OY O1 45 w No IS. O x0 0 - O0 O1 4 C) IN BS 
+ 


7 

18 

19 # If no escaped parentheses... 

20 #+ then 'expr' converts the string operand to an integer. 

231 

22 eelac Vienen gu \YSa\Y ne "eru Uma" g Vet’ QU # Length of string 

23 

24 echo "Boe qoi digits at the beginning or Sa is "espe "Sa" s Vgeg]eu,. 
25 

26 # # 
2 

28 echo 

29 
30 echo "The digits at the beginning of \"Sa\" are "expr "$a" ; "\([0-9]*\)" .” 
sil == == 

32 echo Vihe first 7 Characters gt VU Sae" awe esor YSa™ s U"t(osrcosoc AEN E) 

33 ===== == == 

34 Again, escaped parentheses force a substring match. 

35 
36 edno Jike lest 7 heracles (xs USE are enor USU g YEN arotar N T 
37 ==== end of string operator ^^ 

38 (actually means skip over one or more of any characters until specified 
39 #+ substring) 

40 

41 echo 

42 

43 exit 0 


The above script illustrates how expr uses the escaped parentheses -- \( ... \) -- grouping operator in tandem 
with regular expression parsing to match a substring. Here is a another example, this time from "real life." 


# Strip the whitespace from the beginning and end. 
LO DATS" eor VSI DAM” s "I | esjoxaces ||] 353)» ssssuess] 199" 


# From Peter Knowles! "booklistgen.sh" script 
#+ for converting files to Sony Librie/PRS-50X format. 
4 (http://booklistgensh.peterknowles.com) 


Perl, sed, and awk have far superior string parsing facilities. A short sed or awk "subroutine" within a script 
(see Section 35.2) 1s an attractive alternative to expr. 


en (Gi des (EX [m5 [5 


See Section 10.1 for more on using expr in string operations. 
Notes 


[1] Andeven when xargs is not strictly necessary, it can speed up execution of a command involving 
batch-processing of multiple files. 


Prev Home Next 
External Filters, Programs and Up Time / Date Commands 
Commands 


Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Chapter 16. External Filters, Programs and Commands Next 


16.3. Time / Date Commands 


Time/date and timing 


date 


Simply invoked, date prints the date and time to st dout. Where this command gets interesting is in 
its formatting and parsing options. 


Example 16-10. Using date 


1 #!/bin/bash 

2 Exercising the 'date' command 

3 

4 echo "The number of days since the year's beginning is “date -*$j' ." 
5 Needs a leading '+' to invoke formatting. 

6 %j gives day of year. 

7 

8 echo "The number of seconds elapsed since 01/01/1970 is ^date -4$s'." 
9 $s yields number of seconds since "UNIX epoch" began, 
10 #+ but how is this useful? 
dL 

12 prefix-temp 

13 suffix-$(date +%s) 4 The "+%s" option to 'date' is GNU-specific. 
14 filename-$prefix.S$suffix 

15 echo "Temporary filename = $filename" 

16 # It's great for creating "unique and random" temp filenames, 

17 #+ even better than using $$. 

18 

19 # Read the 'date' man page for more formatting options. 
20 
A exe (0) 


The —u option gives the UTC (Universal Coordinated Time). 


bash$ date 
isos Miele 29) 21:9()7 SS imisa 21902 


bash$ date -u 
Sere Wee 30 OWA3sO07 92/2 wae 290 


This option facilitates calculating the time between different dates. 


Example 16-11. Date calculations 


1 #!/bin/bash 

2 4 date-calc.sh 

3 4 Author: Nathan Coulter 

4 # Used in ABS Guide with permission (thanks!). 
5 

6 

7 


MPHR=60 # Minutes per hour. 
HPD=24 # Hours per day. 
8 
9) elites Oe 4 
10 primer "Ss" S Scere =u USING sae) = 
iil $(date -u -d"$CURRENT" +%s) )) 


( 
( 
12 # Sd = day of month. 


43 
14 
LS 
16 
Jy 
18 
19 
20 
gi 
22 
23 
24 
25 
26 
222) 
28 
29 
30 
Sil 
512 
E 
34 
35 
36 
37 
38 
39) 
40 
41 
42 


CURREN =S (dates =H Sel “2007-09-01 L7SSOsAay Vm STEN mU) 
TARGET=$ (dat ur eN 2007-12-25 127307000 Var STEN qu) 
# SF = full date, $T = %H:%M:%S, $N = nanoseconds, %Z = time zone 
aeaea Aaa 20007, Cg " \ 

"S(date -d"S$CURRENT + 

$(( $(diff) /SMPHR /SMPHR /SHPD / 2 )) days" '+%d $B')" 
# $B = name of month ^ halfway 
printf 'was halfway between %s ' "$(date -d"SCURRENT" '+%d $B')" 
primei "wal Sg we "Sees ge Teel Eb) us 
printf '\nOn $s at $s, there were\n' \ 

$ (date -u -d"SCURRENT" +3F) $(date -u -d"S$CURRENT" +%T) 
DAYS-$(( $(diff) / SMPHR / $MPHR / $HPD )) 
CURRENT-$ (date -d"S$CURRENT +S$DAYS days" '+3F $T.$N $Z') 
HOURS-$(( $(diff) / SMPHR / SMPHR )) 
CURRENT-$ (date -d"SCURRENT +S$HOURS hours" '+%F %T.%N $2Z') 
MINUTES-$(( $(diff) / SMPHR )) 
CURRENT-$ (date -d"SCURRENT -$MINUTES minutes" '+%F £$T.£N $2Z') 
primer Se cays, Se Toucey 9 WSSDENCISU "SIONIS 
joxcubiaWgit Ye Mhault, wol SS Secondes "  "SMDUNUUSS" "US (ela ieie)) A 
printf ‘until Christmas Dinner!\n\n!' 
# Exercise: 
# mE 
# Rewrite the diff () function to accept passed parameters, 
#+ rather than using global variables. 


The date command has quite a number of output options. For example $N gives the nanosecond 
portion of the current time. One interesting use for this is to generate random integers. 


il 
2 
3 
4 
9 
6 
j 
8 


9 


dat 


# 
# 
#+ 


# 1 
# 6 
d 3 


e +3N | sed -e 's/000$//' -e 's/*0//' 
Strip off leading and trailing zeroes, 


Length of generated integer depends on 
how many zeroes stripped off. 


15281052 
3408725 
94504284 


There are many more options (try man date). 


Hi 
2 
3 
4 
5 
6 
j 
8 
9 
LO 
L1 
12 
L3 
L4 
15 
L6 
L7 


dat 
# E 


dat 


# Echoes hour and minute in 24-hour format, 


e +5j 
choes day of the year (days elapsed sin 
e +%k%M 


if present. 


ce January 1). 


as a single digit string. 


(CAOT El waa.) ) 


# The 'TZ' parameter permits overriding the default time zone. 
date 4 Mon Mar 28 21:42:16 MST 2005 

TZ-EST date 4 Mon Mar 28 23:42:16 EST 2005 

# Thanks, Frank Kannemann and Pete Sjoberg, for the tip. 
SixDaysAgo=$ (dat date-'6 days ago") 

OneMonthAgo-$ (dat date-'1 month ago') # Four weeks back 
OneYearAgo=S (dat date='1 year ago') 


zdump 


time 


touch 


at 


batch 


See also Example 3-4 and Example A-43. 
Time zone dump: echoes the time in a specified time zone. 


bash$ zdump EST 
Kol MUG sep le 22: 09327 92.0005 ror 


Outputs verbose timing statistics for executing a command. 


time ls -1 / gives something like this: 


real 0m0.067s 
user 0m0.004s 
sys 0m0.005s 


See also the very similar times command in the previous section. 


c) As of version 2.0 of Bash, time became a shell reserved word, with slightly altered 
behavior in a pipeline. 


Utility for updating access/modification times of a file to current system time or other specified time, 
but also useful for creating a new file. The command touch zzz will create a new file of zero 
length, named zzz, assuming that zzz did not previously exist. Time-stamping empty files in this 
way is useful for storing date information, for example in keeping track of modification times on a 
project. 


8^; The touch command is equivalent to : >> newfileor»» newfile (for 
ordinary files). 

į ) Before doing a cp -u (copy/update), use touch to update the time stamp of files you 
don't wish overwritten. 


As an example, if the directory /home/bozo/tax audit contains the files 
spreadsheet-051606.data, spreadsheet-051706.data, and 
spreadsheet-—051806.data, then doing a touch spreadsheet*.data will protect 
these files from being overwritten by files with the same names during a cp -u 
/home/bozo/financial info/spreadsheet*data /home/bozo/tax audit. 


The at job control command executes a given set of commands at a specified time. Superficially, it 
resembles cron, however, at is chiefly useful for one-time execution of a command set. 


at 2pm January 15 prompts for a set of commands to execute at that time. These commands 
should be shell-script compatible, since, for all practical purposes, the user is typing in an executable 
shell script a line at a time. Input terminates with a Ctl-D. 


Using either the -f option or input redirection (<), at reads a command list from a file. This file is an 
executable shell script, though it should, of course, be non-interactive. Particularly clever is including 
the run-parts command in the file to execute a different set of scripts. 


bash$ at 2:30 am Friday < at-jobs.list 
JOD 2 aie ZOW0—-10—27 0727:30 


The batch job control command is similar to at, but it runs a command list when the system load 
drops below . 8. Like at, it can read commands from a file with the -f option. 


The concept of batch processing dates back to the era of mainframe computers. It means running a 


set of commands without user intervention. 


cal 
Prints a neatly formatted monthly calendar to stdout. Will do current year or a large range of past 
and future years. 
sleep 
This is the shell equivalent of a wait loop. It pauses for a specified number of seconds, doing nothing. 
It can be useful for timing or in processes running in the background, checking for a specific event 
every so often (polling), as in Example 31-6. 
1 sleep 3 # Pauses 3 seconds. 
$^; The sleep command defaults to seconds, but minute, hours, or days may also be 
specified. 
1 sleep 3 h # Pauses 3 hours! 
8^; The watch command may be a better choice than sleep for running commands at 
timed intervals. 
usleep 


Microsleep (the u may be read as the Greek mu, or micro- prefix). This is the same as sleep, above, 
but "sleeps" in microsecond intervals. It can be used for fine-grained timing, or for polling an ongoing 
process at very frequent intervals. 


1 usleep 30 # Pauses 30 microseconds. 


This command is part of the Red Hat initscripts / rc-scripts package. 


The usleep command does not provide particularly accurate timing, and is therefore 
unsuitable for critical timing loops. 
hwclock, clock 
The hwclock command accesses or adjusts the machine's hardware clock. Some options require root 
privileges. The /etc/rc.d/rc.sysinit startup file uses hwclock to set the system time from 
the hardware clock at bootup. 


The clock command is a synonym for hwclock. 
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16.4. Text Processing Commands 


Commands affecting text and text files 


sort 


tsort 


uniq 


File sort utility, often used as a filter in a pipe. This command sorts a text stream or file forwards or 
backwards, or according to various keys or character positions. Using the -m option, it merges 
presorted input files. The info page lists its many capabilities and options. See Example 11-9, 


Example 11-10, and Example A-8. 

Topological sort, reading in pairs of whitespace-separated strings and sorting according to input 
patterns. The original purpose of tsort was to sort a list of dependencies for an obsolete version of the 
ld linker in an "ancient" version of UNIX. 

The results of a tsort will usually differ markedly from those of the standard sort command, above. 


This filter removes duplicate lines from a sorted file. It is often seen in a pipe coupled with sort. 


i es lagt-i lisit-2 list-s3 | sort | Wang > timal. bisg 
2 4 Concatenates the list files, 

3 # sorts them, 
4 d 
E 


removes duplicate lines, 
# and finally writes the result to an output file. 


The useful -c option prefixes each line of the input file with its number of occurrences. 


bash$ cat testfile 
This line occurs only once. 
Haas lina CEUS EWES. 
This line occurs twice. 
This line occurs three times. 
This line occurs three times. 
Mase line cetrs TEloustexe Times, 


bash$ uniq -c testfile 
i twe line oceurcs cul once. 
2 This line occurs twice. 
3 This line occurs three times. 


bash$ sort testfile | uniq -c | sort -nr 
3 This line occurs three times. 
2 Wins Lina Geewues) CHLOE, 
1 This line occurs only once. 


The sort INPUTFILE | uniq -c | sort -nr command string produces a frequency of 
occurrence listing on the INPUTFILE file (the -nr options to sort cause a reverse numerical sort). 
This template finds use in analysis of log files and dictionary lists, and wherever the lexical structure 
of a document needs to be examined. 


Example 16-12. Word Frequency Analysis 


#!/bin/bash 
# wf.sh: Crude word frequency analysis on a text file. 
# This is a mor HILCIEINE Versio Cie igne yrz cin”! series . 


(Gai dE (G8) SS) J 


6 Check for input file on command-line. 
7 ARGS-1 
8 E BADARGS-85 
9 E NOFILE-86 
10 
11 if [ $4 -ne "SARGS" ] # Correct number of arguments passed to script? 
12 then 
L3) echo "Usage: 'basename $0? filename" 
14 exit $E BADARGS 
TORTI 
16 
J7 asc | 9$ em "SL" | # Check if file exists. 
18 then 
19 echo "File N"$1N" does not exist." 
20 exit $E NOFILE 
2L ae ak 
22 
25 
24 
25 FHEREEE EEE EE HEHE HE EEE EE HH HE EEE EE HH EE EE ER EHR EE EH 
26 main () 
27 sed -e 's/N.//g' =e 's/N,//g' -e 's/ /\ 
28 Jg" WSIW | sque UXX-EU Hazy | eoret | umiec -e | cort -A7 
29 
30 Frequency of occurrence 
Sal 
92) Filter out periods and commas, and 
33 #+ change space between words to linefeed, 
34 #+ then shift characters to lowercase, and 
35 #+ finally prefix occurrence count and sort numerically. 
36 
37 Arun Giridhar suggests modifying the above to: 
38 | Some || take -€ | sore ail =E] | Sex s mu 
S99 This adds a secondary sort key, so instances of 
40 #+ equal occurrence are sorted alphabetically. 
41 As he explains it: 
42 "This is effectively a radix sort, first on the 
43 #+ least significant column 
44 #+ (word or string, optionally case-insensitive) 
45 #+ and last on the most significant column (frequency)." 
46 
47 As Frank Wang explains, the above is equivalent to 
48 #+ | sore | wie =e | sort +0 mr 
49 #+ and the following also works: 
50 #+ "ENS | Sexe | mue; =e | Some kilmei =k 
Sil HPT HE EEE EE EEE HE EE EH EE EH HE EE EEE EE HE HE EEE RE EE EE EEE 
52 
53 exit 0 
54 
55 # Exercises: 
ONE aaa 
57 # 1) Add 'sed' commands to filter out other punctuation, 
58 #+ such as semicolons. 
59 # 2) Modify the script to also filter out multiple spaces and 
60 #+ other whitespace. 
bash$ cat testfile 
This line occurs only once. 
This line occurs twice. 
This line occurs twice. 
This line occurs three times. 
This line occurs three times. 
This line occurs three times. 


bash$ ./wf.sh testfile 
this 

occurs 

line 

times 

three 

twice 

only 

once 


H = ISS) (09) 163) (ex, tery re») 


expand, unexpand 


cut 


The expand filter converts tabs to spaces. It is often used in a pipe. 


The unexpand filter converts spaces to tabs. This reverses the effect of expand. 


A tool for extracting fields from files. It is similar to the print $N command set in awk, but more 
limited. It may be simpler to use cut in a script than awk. Particularly important are the —d (delimiter) 


and —f (field specifier) options. 
Using cut to obtain a listing of the mounted filesystems: 


i, qug scl US U 15,2 uec nnl 
Using cut to list the OS and kernel version: 


dL unene -A | cuu acl? Y —i1,39,13,2 
Using cut to extract message headers from an e-mail folder: 


bash$ grep '^Subject:' read-messages | cut -c10-80 
Re: Linux suitable for mission-critical apps? 
MAKE MILLIONS WORKING AT HOME!!! 

Spam complaint 

Re: Spam complaint 


Using cut to parse a file: 


1 4$ List all the users in /etc/passwd. 
2 
3 FILENAME-/etc/passwd 
4 
5 for user in $(cut a: —fl SFILENAME) 
6 do 
7 echo $user 
8 done 
B 
10 # Thanks, Oleg Philon for suggesting this. 
cut -d ' ' -£2,3 filename is equivalent to awk -F'[ ]' '( print $2, 
filename 


8^; It is even possible to specify a linefeed as a delimiter. The trick is to actually embed a 


linefeed (RETURN) in the command sequence. 


bash$ cut -d' 

' -£3,7,19 testfile 

Wows as) inne 3 Oi euge. 
inis sues lima 7 ou esie. 
Wows is line L9 qus jeSsicitalile. 


Thank you, Jaka Kranjc, for pointing this out. 
See also Example 16-48. 


$3 }' 


paste 


join 


head 


Tool for merging together different files into a single, multi-column file. In combination with cut, 
useful for creating system log files. 


Consider this a special-purpose cousin of paste. This powerful utility allows merging two files in a 
meaningful fashion, which essentially creates a simple version of a relational database. 


The join command operates on exactly two files, but pastes together only those lines with a common 
tagged field (usually a numerical label), and writes the result to stdout. The files to be joined 
should be sorted according to the tagged field for the matchups to work properly. 


wales 1L elaia 


200 Laces 


il 
2 
3 100 Shoes 
4 
5 S00 Socks 


wiles 2. claica 


100 $40.00 
200 $1.00 
300 $2.00 


(nl dE (3 [say (3 


bash$ join 1.data 2.data 
File: 1.data 2.data 


100 Shoes $40.00 


200 Laces $1.00 
300 Socks) $2.00 


$^; The tagged field appears only once in the output. 


lists the beginning of a file to stdout. The default is 10 lines, but a different number can be 
specified. The command has a number of interesting options. 


Example 16-13. Which files are scripts? 


1 #!/bin/bash 

2 4 script-detector.sh: Detects scripts within a directory. 
3 

4 TESTCHARS=2 # Test first 2 characters. 

5 SHABANG-'j4!' 4 Scripts begin with a "sha-bang." 

6 

7 for file in * # Traverse all the files in current directory. 
8 do 

S if [I "head -c$TESTCHARS "$file"'^ = "SSHABANG" ]] 

10 # head -c2 #! 

il # The '-c' option to "head" outputs a specified 

2 #+ number of characters, rather than lines (the default). 
3 then 

4 eho "iple Werle ale a srexealjore . 

5 else 

6 eao Wailea WETE U la mot a Seriot. UU 

7 $E al 

18 done 

19 

20 exit 0 

Bil 


22 # Exercises: 


2g do f————————9— 

24 1) Modify this script to take as an optional argument 

25 #+ the directory to scan for scripts 

ZONE (rather than just the current working directory). 

2: 

28 2) As it stands, this script gives "false positives" for 
29 #+ Perl, awk, and other scripting language scripts. 

30 Comeacice class. 


Example 16-14. Generating 10-digit random numbers 


!/bin/bash 
rnd.sh: Outputs a 10-digit random number 


Script by Stephane Chazelas. 


head -c4 /dev/urandom | od -N4 -tu4 | sed -ne '1s/.* //p' 


Analysis 


head: 
-c4 option takes first 4 bytes. 


od: 
-NA option limits output to 4 bytes. 
-tu4 option selects unsigned decimal format for output. 


NPRPRPPRP PPP PY 
SCOMIDGDHRWNHFPOWOMIDGURWNHE 


2L sed: 
22 -n option, in combination with "p" flag to the "s" command, 
2,9) outputs only matched lines. 
24 
29 
26 
2:1) The author of this script explains the action of 'sed', as follows. 
28 

haal -64 Jelew/bnseuelown || @cl -N4 Scud | exl sae "de. Jf" 

> | 
ASSUMNE GxburjswhE Uo io. "eel ——————-—-— M 


is 0000000 1198195154^n 


sed begins reading characters: 0000000 1198195154\n. 
Here it finds a newline character, 

+ so it is ready to process the first line (0000000 1198195154). 
It looks at its «range»«action»s. The first and only one is 


CO CO CO CO CO CO CO CO CO CO Dd 
We) Go) | (ex. Gal mw TOS [So [LV e» Ke) 


40 range action 

41 il E sw Jf 

42 

45 The line number is in the range, so it executes the action: 

44 #+ tries to substitute the longest string ending with a space in the line 
45 (VMOOOOOOM YW) wira mowmine (/ 7), ane abit Sti succede, Drine iis Cestit 
46 ("p" is a flag to the "s" command here, this is different 

47 #+ from the "p" command). 

48 

49 sed is now ready to continue reading its input. (Note that before 

50 #+ continuing, if -n option had not been passed, sed would have printed 
51 #+ the line once again). 


tail 


52 
33 
54 
339 
56 
57 
58 
3$ 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
ya 
Y2 
WS 
74 
TS 
76 
77] 
78 
TS 
80 
SL 
82 


Now, sed reads the remainder of the characters, and finds the 
+ end of the file. 
It is now ready to process its 2nd line (which is also numbered '$' as 
ar aie Ss the last ome) s 

It sees it is not matched by any <range>, so its job is done. 


In few word this sed commmand means: 
"On the first line only, remove any character up to the right-most space, 
ap eleven Prine alice... 


A better way to do this would have been: 
eed =e 's/.* fpe 


Here, two «range»«action»s (could have been written 
eed =e "s/.^ f/f" -a sr 


range action 
nothing (matches line) SUA 
nothing (matches line) er (emt) 


Here, sed only reads its first line of input. 
It performs both actions, and prints the line (substituted) before 
* quitting (because of the "q" action) since the "-n" option is not passed. 


# 


An even simpler altenative to the above one-line script would be: 
head -c4 /dev/urandom| od -An -tu4 


exit 


See also Example 16-39. 


lists the (tail) end of a file to stdout. The default is 10 lines, but this can be changed with the -n 
option. Commonly used to keep track of changes to a system logfile, using the -f option, which 
outputs lines appended to the file. 


Example 16-15. Using fail to monitor the system log 


jh 
(t We) (e| sa) rex; Gal de» GH [SS fe 


pes 
= 


| o mbpgo 
awe Wd 


#!/bin/bash 

filename=sys.log 

cat /dev/null > $filename; echo "Creating / cleaning out file." 
# Creates file if it does not already exist, 

#+ and truncates it to zero length if it does. 


# : > filename and > filename also work. 


tail /var/log/messages > $filename 
# /var/log/messages must have world read permission for this to work. 


echo "$filename contains tail end of system log." 


esate d) 


į ) To list a specific line of a text file, pipe the output of head to tail -n 1. For example 
head -n 8 database.txt | tail -n 1 lists the 8th line of the file 
database.txt. 


grep 


To set a variable to a given block of a text file: 


var-$(head m $m S$filename | tail -n $n) 


4 m = from beginning of file, number of lines to end of block 


i 

2 

3 # filename = name of file 

4 

5 $n = number of lines to set variable to (trim from end of block) 


«eg; Newer implementations of tail deprecate the older tail -SLINES filename usage. The 
standard tail -n $LINES filename is correct. 


See also Example 16-5, Example 16-39 and Example 31-6. 


A multi-purpose file search tool that uses Regular Expressions. It was originally a command/filter in 
the venerable ed line editor: g/re/p -- global - regular expression - print. 


grep pattern[file...] 


Search the target file(s) for occurrences of pattern, where pattern may be literal text or a 
Regular Expression. 


bash$ grep '[rst]ystem.$' osinfo.txt 
The GPL governs the distribution of the Linux operating system. 


If no target file(s) specified, grep works as a filter on stdout, as in a pipe. 


bash$ ps ax | grep clock 
TES ted S 0:00 xclock 
SOIL jonessi/ 1L S 0200 creo Clock 


The —i option causes a case-insensitive search. 
The —w option matches only whole words. 
The —1 option lists only the files in which matches were found, but not the matching lines. 


The —r (recursive) option searches files in the current working directory and all subdirectories below 
it. 


The —n option lists the matching lines, together with line numbers. 


bash$ grep -n Linux osinfo.txt 
2:This is a file containing information about Linux. 
6:The GPL governs the distribution of the Linux operating system. 


The -v (or --invert-match) option filters out matches. 


1 grep patternl *.txt | grep -v pattern2 

2 

3 4 Matches all lines in "*.txt" files containing "patternl", 
A ae leyung “sino WVeercieeiein2 c 


The -c (--count) option gives a numerical count of matches, rather than actually listing the 
matches. 


grag -© ioe ~~ 5 Siepnul # (number of occurrences of "txt" in "*.sgml" files) 


(ni dex (es) Iss) [S 


# means count (-c) zero-separated (-z) items matching "." 

# that is, non-empty ones (containing at least 1 character). 

# 

printf 'a bNnc d\n\n\n\n\n\000\n\000e\000\000\nf' | grep -cz # 3 
printf 'a bNnc d\n\n\n\n\n\000\n\000e\000\000\nf' | grep -cz '$' # 5 
printf 'a bNnc d\n\n\n\n\n\000\n\000e\000\000\nf' | grep -cz '^' #5 


printf 'a bNnc CAN aAA OCTAN A C0 CTE 0T TAC CON a | grep -c '$' # 9 
By default, newline chars (\n) separate items to match. 


Note that the -z option is GNU "grep" specific. 


Wey (0s) «| ep) (On des 169 IS) [= «9 We) (sex <a] O5 


il Tis ou tan 

The --color (or --colour) option marks the matching string in color (on the console or in an 
xterm window). Since grep prints out each entire line containing the matching pattern, this lets you 
see exactly what is being matched. See also the -o option, which shows only the matching portion of 
the line(s). 


Example 16-16. Printing out the From lines in stored e-mail messages 


#!/bin/bash 
# from.sh 


# Emulates the useful "from" utility in Solaris, BSD, etc. 
# Echoes the "From" header line in all messages 
#+ in your e-mail directory. 


NPRPRPPRPP PP PY 
O (o 0» -1 O Oi i$ (). M PS. O xo 0 -1 O O1 4 CQ I0 2 


MAILDIR--/mail/* No quoting of variable. Why? 
GREE OPTSS"-H =A 5 --cologr" Show file, plus extra context lines 
+ and display "From" in color. 
TARGETSTR-"^From" "From" at beginning of line. 
for file in SMAILDIR No quoting of variable. 
do 
grep S$GREP OPTS "STARGETSTR" "Sfile" 
# GER SEN QE Again, do not quote this variable. 
echo 
done 
21 eua S? 
22 
23 1 Wireline wiem co oios Cas gu gi (tse seruit t9 "were" ox 


N 
[x 


#+ redirect it to a file 


When invoked with more than one target file given, grep specifies which file contains matches. 


bash$ grep Linux osinfo.txt misc.txt 
osinfo.txt:This is a file containing information about Linux. 
osinfo.txt:The GPL governs the distribution of the Linux operating system. 
misc.txt:The Linux operating system is steadily gaining in popularity. 


į ) To force grep to show the filename when searching only one target file, simply give 
/ dev/nul1l as the second file. 


bash$ grep Linux osinfo.txt /dev/null 
OSIM GEME SAL ihe: Gl AIS Conec kae MEO MAMEIESLOIN Clore U 
osinfo.txt:The GPL governs the distribution of the Linux operating system. 


If there is a successful match, grep returns an exit status of 0, which makes it useful in a condition test 
in a script, especially in combination with the -q option to suppress output. 


1 SUCCESS-0 # if grep lookup succeeds 

2 word=Linux 

3 filename-data.file 

4 

5 grep -q "Sword" "$filename" # The "-q" option 

6 #+ causes nothing to echo to stdout. 
7 ix | SP eee SSUCCESS | 

8 # if grep -q "Sword" "Sfilename" can replace lines 5 E 
9 then 
10 echo "Sword found in $filename" 
11 else 
12 echo "Sword not found in $filename" 
U3) sta 


Example 31-6 demonstrates how to use grep to search for a word pattern in a system logfile. 


Example 16-17. Emulating grep in a script 


!/bin/bash 
grp.sh: Rudimentary reimplementation of grep. 


E BADARGS-85 


za [ =e SE" ] # Check for argument to script. 
then 

echo "Usage: 'basename $0' pattern" 

exit $E BADARGS 
fti 


echo 


itus saile abi 2 # Traverse all files in SPWD. 
do 
output-$(sed -n /"$1"/p $file) # Command substitution. 


iid [ | =x VSowuwtput" | # What happens if "Soutput" is not quoted? 
then 

echo n "$file: " 

echo "Soutput" 
fta # sed -ne "/$1/s|*|${file}: |p" is equivalent to above. 


[3 [S3 [RS MS i mE ee eee ees 
(eS) [ey [5 c9» (er (Ger c ey) (Gn PS (EX) [my [— 695 We) (09r c ep) (Gn BSS (es) [sy [5 


N 
od 


echo 
done 


NNN 
~ O (nl 


echo 


N 
[99] 


exse O 


w CO N 
i Do 


# Exercises: 


Ww 
N 
+ 


33 # 1) Add newlines to output, if more than one match in any given file. 
34 # 2) Add features. 


How can grep search for two (or more) separate patterns? What if you want grep to display all lines 
in a file or files that contain both "pattern1" and "pattern2"? 


One method is to pipe the result of grep pattern1 to grep pattern2. 


For example, given the following file: 


i 
2 
3 
4 
5 
6 
7 


Now, let's search this file for lines containing both "file" and "text"... 


bash$ 
# FL 
This 
This 
This 
This 


This 
This 


# Filename: tstfile 


This is a sample file. 

Ius ae Elin Oriecluneiey Cee LSe 

This file does not contain any unusual text. 
aes Le die: MONE MISE . 

Here is some text. 


grep file tstfile 
lename: tstfile 

is a sample file. 

is an ordinary text file. 

file does not contain any unusual text. 
eils Is Nor Waustied . 


bash$ grep file tstfile | grep text 


is an ordinary text file. 
file does not contain any unusual text. 


Now, for an interesting recreational use of grep... 


Example 16-18. Crossword puzzle solver 


NPRPRPPRPP PP PY 
O (o 00 -1 O Oi i$ (). M PB. O xo 0 -1 O O1 4 CQ Io E 


NNN NO 
Oe WN ER 


26 
27 
28 
29 
30 
snl 
22 
33 
34 
35 
36 
Shy) 
38 
3 
40 


!/bin/bash 
cw-solver.sh 


+ so you need a list of all valid words 
+ with the known letters in given positions. 
Hor exampiltec wie 

192 05 9? 9 210 
W iim pOSte © dL, 3 WMalkinewing, 3L sum Chs SUB 
(See comments at end of script.) 


. NOPATT-71 
ICT-/usr/share/dict/word.lst 


A^AA^AA^A^A^A^A^^ 


U m 


ASCII word list, one word per line. 
If you happen to need an appropriate list, 


http://ibiblio.org/pub/Linux/libs/yawl-0.3.2.tar.gz 
or 
http://bash.neuralshortcircuit.com/yawl-0.3.2.tar.gz 
en =a UEU # If no word pattern specified 
then #+ as a command-line argument 
echo mr 6 a e Enen 
echo "Usage:" #+ Usage message. 
echo 
Gemo VWUSOur \Wioercieeyeia,, AN UU 
echo "where \"pattern\" is in the form" 
eohosxcc AoC am ey 
echo 
echo "The x's represent known letters," 
echo "and the periods are unknown letters (blanks) ." 
echo "Letters and periods can be in any position." 
echo "For example, try: Sin CuSO Sin Wed oiloao cin! 
echo 
exit SE_NOPATT 


This is actually a wrapper around a one-liner 


Crossword puzzle and anagramming word game solver. 
You know *some* of the letters in the word you're looking for, 


4 unknowns, 


+ download the author's "yawl" word list package. 


Looks for word list here. 


(line 46). 


n at the end. 


A EL 
42 

43 echo 
44 
45 This is where all the work gets done. 

4G grea ^USIWS WS # Yes, only one line! 
47 | | 

48 ^ is start-of-word regex anchor. 

49 $ is end-of-word regex anchor. 

50 
yal prem Suse! Creo Wieleks o vol. i, 

52 #+ a book the ABS Guide author may yet get around 
53 #+ to writing . . . one of these days 

54 
55 echo 

56 

57 

58 exit $? # Script terminates her 

59 # If there are too many words generated, 
60 #+ redirect the output to a file. 

61 

62 $ sim cgu-solwer:sm Wo eoteco old 

63 

64 wellington 

65 workingman 

66 workingmen 


egrep -- extended grep -- is the same as grep -E. This uses a somewhat different, extended set of 
Regular Expressions, which can make the search a bit more flexible. It also allows the boolean | (or) 
operator. 


bash $ egrep 'matches|Matches' file.txt 
Line 1 matches. 

Line 3 Matches. 

Line 4 contains matches, but also Matches 


fgrep -- fast grep -- is the same as grep -F. It does a literal string search (no Regular Expressions), 
which generally speeds things up a bit. 


$^; On some Linux distros, egrep and fgrep are symbolic links to, or aliases for 
grep, but invoked with the -E and -F options, respectively. 


Example 16-19. Looking up definitions in Webster's 1913 Dictionary 


! /bin/bash 
dict-lookup.sh 
This script looks up definitions in the 1913 Webster's Dictionary. 
This Public Domain dictionary is available for download 
+ from various sites, including 
+ Project Gutenberg (http://www.gutenberg.org/etext/247). 


Convert it from DOS to UNIX format (with only LF at end of line) 
4r Dekore Vonne GE WIE! Caig SOILS s 

Store the file in plain, uncompressed ASCII text. 

Set DEFAULT DICTFILE variable below to path/filename. 


E BADARGS-85 
AXCONTEXTLINES=50 # Maximum number of lines to show. 
DEFAULT_DICTFILE="/usr/share/dict/webster1913-dict .txt" 


=<] deny (nl des (65) > [9 eS) We) (ex c] ey) (nl gEx (es) Is) [5 


18 
ALS, 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
Sul 
32 
33 
34 
39 
36 
37 
38 
EX 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Syl 
52 
53 
54 
59 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
ak 
12 
73 
74 
353) 
76 
Vd 
78 
WS 
80 
81 
82 
83 


# Default dictionary file pathname. 
# Change this as necessary. 

# Note 

# PER ee 

# This particular edition of the 1913 Webster's 

#+ begins each entry with an uppercase letter 

#+ (lowercase for the remaining characters). 


# Only the *very first line* of an entry begins this way, 
#+ and that's why the search algorithm below works. 


aie [| —4 Secho "SIL" | sec =a "/^li—21)49"») 11 
# Must at least specify word to look up, and 
#+ it must start with an uppercase letter. 


then 
echo "Usage: ~basename $0' Word-to-define [dictionary-file]" 
echo 
echo "Note: Word to look up must start with capital letter," 
echo "with the rest of the word in lowercase." 
Chom " 
echo "Examples: Abandon, Dictionary, Marking, etc." 
exit $E BADARGS 
imal 
sue [ eu Z # May specify different dictionary 
#+ as an argument to this script. 
then 
dictfile-$DEFAULT DICTFILE 
else 
dictfile-"$2" 
E: 
Definition-$(fgrep -A S$MAXCONTEXTLINES "$1 NX" "Sdictfile") 


Dieux E3umnes 33m orn Morel oor” 


And, yes, "fgrep" is fast enough 
+ to search even a very large text file. 


Now, snip out just the definition block. 


seko VSibyesrskamiesem” | 

sec Sa Tip, / ASB [ 

# Print from first line of output 

#+ to the first line of the next entry. 

sed 'Sd' | sed 'Sd' 

# Delete last two lines of output 

#+ (blank line and first line of next entry). 
# 


Goeie SP 


Exercises: 

1) Modify the script to accept any type of alphabetic input 
+ (uppercase, lowercase, mixed case), and convert it 

+ to an acceptable format for processing. 


2) Convert the script to a GUI application, 
+ using something like 'gdialog' or 'zenity' 
The script will then no longer take its argument (s) 
+ from the command-line. 


look 


84 # 3) Modify the script to parse one of the other available 
85 4 * Public Domain Dictionaries, such as the U.S. Census Bureau Gazetteer. 


8^; See also Example A-41 for an example of speedy fgrep lookup on a large text file. 


agrep (approximate grep) extends the capabilities of grep to approximate matching. The search string 
may differ by a specified number of characters from the resulting matches. This utility is not part of 
the core Linux distribution. 


j ) To search compressed files, use zgrep, zegrep, or zfgrep. These also work on 
non-compressed files, though slower than plain grep, egrep, fgrep. They are handy 
for searching through a mixed set of files, some compressed, some not. 


To search bzipped files, use bzgrep. 


The command look works like grep, but does a lookup on a "dictionary," a sorted word list. By 
default, look searches for a match in /usr/dict/words, but a different dictionary file may be 
specified. 


Example 16-20. Checking words in a list for validity 


1 #!/bin/bash 
2 4 lookup: Does a dictionary lookup on each word in a data file. 
E 
4 file-words.data # Data file from which to read words to test. 
5 
6 echo 
7 
8 while [ "Sword" != end ] # Last word in data file. 
DECIS gn One 
LO read word # From data file, because of redirection at end of loop. 
Li look Sword > /dev/null # Don't want to display lines in dictionary file. 
12 lookup-$? # Exit status of 'look' command. 
13 
14 ai | YSlocktp” -ee 0 | 
LS then 
16 Seno AUSTAS aber yaelllsicl., w 
Ly else 
iL Geng VW Soc ie imyalici, ” 
1g iE a 
20 
2L come «UI s lebe stein CO Sil. so Vreacks! Come Erom (Eee. 
22 
23 echo 
24 
25 exit 0 
26 
27 # 
28 4 Code below line will not execute because of "exit" command above. 
29 
30 
31 # Stephane Chazelas proposes the following, more concise alternative: 
32 
33 while read word && [[ $word != end ]] 
S4 dgio at look Tvora > Jokes 
35 ies Gxelao UNUS worod Jue Seuil; 


36 else echo “\"Sworcd\Y ig invalle" 


37 iE aL 

38 done <"Sfile" 
39 

40) exui 10) 


sed, awk 
Scripting languages especially suited for parsing text files and command output. May be embedded 
singly or in combination in pipes and shell scripts. 


sed 
Non-interactive "stream editor", permits using many ex commands in batch mode. It finds many uses 
in shell scripts. 

awk 
Programmable file extractor and formatter, good for manipulating and/or extracting fields (columns) 
in structured text files. Its syntax is similar to C. 

we 


wc gives a "word count" on a file or I/O stream: 


bash $ wc /usr/share/doc/sed-4.1.2/README 
13 70 447 README 
[13 lines 70 words 447 characters] 


wc —w gives only the word count. 

wc —1 gives only the line count. 

wc —c gives only the byte count. 

wc —m gives only the character count. 

wc -L gives only the length of the longest line. 


Using we to count how many .t xt files are in current working directory: 


il S le + wtar || we e 

2 Will work as long as none of the "*.txt" files 

3 #+ have a linefeed embedded in their name. 

4 

3 Alternative ways of doing this are: 

6 pine 5 =meseclejsicia i -nams Yro -print || grab ce 
7 (shope =$ mullal: ger == = ta Echo S) 

8 

9 


Thanks, S.C. 
Using we to total up the size of all the files whose names begin with letters in the range d - h 


bash$ wc [d-h]* | grep total | awk '(print $3)' 
MBSA 


Using wc to count the instances of the word "Linux" in the main source file for this book. 


bash$ grep Linux abs-book.sgml | wc -1 
50 


See also Example 16-39 and Example 20-8. 
Certain commands include some of the functionality of wc as options. 


iL soo [| orep dex || vue l 
2 # This frequently used construct can be more concisely rendered. 


tr 


3 
4 Iigrep cT too 

5 4 Just use the "-c" (or "--count") option of grep. 
6 

7 


ie ieu sen. Sacs 
character translation filter. 
D Must use quoting and/or brackets, as appropriate. Quotes prevent the shell from 


reinterpreting the special characters in tr command sequences. Brackets should be 
quoted to prevent expansion by the shell. 


Either tr "A-Z" "*" <filenameortr A-Z \* «filename changes all the uppercase 
letters in filename to asterisks (writes to stdout). On some systems this may not work, but tr 
A-Z '[**]' will. 


The —d option deletes a range of characters. 


echo "abcdef" # abcdef 
echo "abcdef" | tr -d b-d # aef 


ie =—Cl (Q9 «eise 
# Deletes all digits from the file "filename". 


The --squeeze-repeats (or —s) option deletes all but the first instance of a string of 
consecutive characters. This option is useful for removing excess whitespace. 


(ex (Gal dex (69. [soy p 


bash$ echo "XXXXX" | tr --squeeze-repeats 'X' 
X 


The -c "complement" option inverts the character set to match. With this option, tr acts only upon 
those characters not matching the specified set. 


bash$ echo "acfdeb123" | tr -c b-d + 
+e+d+b++++ 


Note that tr recognizes POSIX character classes. [1] 


bash$ echo "abcd2ef1" | tr '[:alpha:]' - 
----2--1 


Example 16-21. toupper: Transforms a file to all uppercase. 


! /bin/bash 
Changes a file to all uppercase. 


E BADARGS-85 
abi p xe WW ] # Standard check for command-line arg. 


echo "Usage: “basename $0! filename" 
E BADARGS 


is uum Jw! seg 


| omopmo 
(Q N HP O0 -J1o0 50 ND 
[0] 
x 
H 
es 
ny 


14 # Same effect as above, but using POSIX character set notation: 
# xe "[slexsess]" "[suppesss]l" «"S1" 
Jb Ge Thanks, Sao 


[En 
[91] 


20 # Exercise: 
21 # Rewrite this script to give the option of changing a file 
22 #+ to *either* upper or lowercase. 


Example 16-22. lowercase: Changes all filenames in working directory to lowercase. 


1 #!/bin/bash 

2 t 

3 # Changes every filename in working directory to all lowercase. 
4 d 

5 # Inspired by a script of John Dubois, 

6 #+ which was translated into Bash by Chet Ramey, 

7 #+ and considerably simplified by the author of the ABS Guide. 

8 

9 
10 for filename in * # Traverse all files in directory. 
JL3b dle 
12 fname= basename $filename” 
13 n= echo Sfname | tr A-Z a-z # Change name to lowercase. 

14 if [| "Sfname" != "Sn" ] # Rename only files not already lowercase. 
15 then 

16 mv $fname $n 

alley) fes 

18 done 

19 
20 ex S 
21 
22 
23 # Code below this line will not execute because of "exit". 
24 # # 
25 # To run it, delete script above line. 
26 


27 # The above script will not work on filenames containing blanks or newlines. 
28 # Stephane Chazelas therefore suggests the following alternative: 


30 

31 for filename in * # Not necessary to use basename, 

32 i Sine Yel vos i. rerea qug ele Comeau WU. 
SS) tolo) sai echo Uie stlbeyavennsy/ 0 | dese upper: a a overa || U^ 

34 # POSIX char set notation. 

SERT Slash added so that trailing newlines are not 

36 # removed by command substitution. 

37 4 Variable substitution: 

38 n=S {n%/} # Removes trailing slash, added above, from filename. 
39 LI Stilenam ==} Sin I| | |) imp YU Sieiemena” Veny 

40 # Checks if filename already lowercase. 

41 done 

42 

a3 Gece Se 


Example 16-23. du: DOS to UNIX text file conversion. 


!/bin/bash 
Du.sh: DOS to UNIX text file converter. 


de eh [9 [p 


E WRONGARGS-65 


(oo) =a) fox, [Ga] ges. Gol Sy T=) «€» Wer feor c Cony Gal 


if [ -z "$q" ] 


echo "Usage: `basename $0` 


exit $E WRONGARGS 


NEWFILENAME-$1.unx 


CR2'N015' 


we xe SOR < Si e sii 


Lines ina 
Lines ina 


Carriage return. 
OLS As OCEAL ASCL COCs ioe Cr. 
DOS text file end in CR-LF. 

UNIX text file end in LF only. 


EWFILENAM 


filename-to-convert" 


# Delete CR's and write to new file. 


echo "Original DOS 


[recte cristo alls) EST oY 


echo "Converted UNIX text file is \"SNEWFILENAME\"." 


exec O 


# Exercise: 


# Change the above script to convert from UNIX to DOS. 


Example 16-24. rot13: ultra-weak encryption. 


1 #!/bin/bash 

2 woOeELS gine Classic otla algorit iain, 

3 encryption that might fool a 3-year old. 

4 

5 Usage: ./rot13.sh filename 

6 or ./rotl3.sh «filename 

Ji or ./rotl3.sh and supply keyboard input (stdin) 

8 

9 cat "S8" | tr 'a-zA-Z' 'n-za-mN-ZA-M' d acl EOS Seats oM do Mtr Mete oe er edm 
10 Ths "cnc WEWE constriction 
11 #+ permits getting input either from stdin or from files. 
12 
13 exit 0 

Example 16-25. Generating "Crypto-Quote" Puzzles 

1 #!/bin/bash 

2 crypto-quote.sh: Encrypt quotes 

3 

4 Will encrypt famous quotes in a simple monoalphabetic substitution. 
5 The result is similar to the "Crypto Quote" puzzles 

6 #+ seen in the Op Ed pages of the Sunday paper. 

7 

8 

9 key-ETAOINSHRDLUBCFGJMQPVWZYXK 
10 The "key" is nothing more than a scrambled alphabet. 
dal Changing the "key" changes the encryption. 
12 
13 The Year "SU KONA UIGIE LSI Gets imge either iron stein (he rron tikes. 
14 If using stdin, terminate input with a Control-D. 
LS) Otherwise, specify filename as command-line parameter. 


fold 


fmt 


16 
Ly 
18 
19 
20 
2L 
22 
23 
24 
25 
26 
2 
28 
28 
30 
Su 
22 
33 
34 


42 
43 


Gu USE ui | EE ma-z" "A-7" | oie "A-Z" US esy t 
| to uppercase | encrypt 
Will work on lowercase, uppercase, or mixed-case quotes. 
Passes non-alphabetic characters through unchanged. 
Try this script with something like: 
"Nothing so needs reforming as other people's habits." 
--Mark Twain 
OMESUTE 395 
"CFPHRCS QF CIIOQ MINFMBRCS EQ FPHIM GIFGUI'Q HETRPQ." 
--BEML PZERC 
To reverse the encryption: 
cat ur fe t | tr "Skey" "a-z" 
This simple-minded cipher can be broken by an average 12-year old 
* using only pencil and paper. 
exit 0 
# Exercise: 


# Modify the script so that it will either encrypt or decrypt, 
#+ depending on command-line argument (s). 


tr variants 


The tr utility has two historic variants. The BSD version does not use brackets (tr a-z A-Z), but 
the SysV one does (tr '[a-z]' '[A-Z] '). The GNU version of tr resembles the BSD one. 


A filter that wraps lines of input to a specified width. This is especially useful with the —s option, 
which breaks lines at word spaces (see Example 16-26 and Example A-1). 


Simple-minded file formatter, used as a filter in a pipe to "wrap" long lines of text output. 


Example 16-26. Formatted file listing. 


w cp wx) (Ga des Ge RS) I 


«o 


10 
deal 
12 


#!/bin/bash 

WIDTH=40 # 40 columns wide. 

b= le /usr/local/bin' # Get a file listing... 
echo $b | fmt -w SWIDTH 


# Could also have been done by 
# echo $b | fold - -s -w SWIDTH 


exit 0 


See also Example 16-5. 


col 


column 


colrm 


į ) A powerful alternative to fmt is Kamil Toman's par utility, available from 
http://www.cs.berkeley.edu/-amc/Par/. 


This deceptively named filter removes reverse line feeds from an input stream. It also attempts to 
replace whitespace with equivalent tabs. The chief use of col is in filtering the output from certain text 
processing utilities, such as groff and tbl. 


Column formatter. This filter transforms list-type text output into a "pretty-printed" table by inserting 
tabs at appropriate places. 


Example 16-27. Using column to format a directory listing 


#!/bin/bash 
# colms.sh 
# A minor modification of th xample file in the "column" man page. 


(printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAME\n" \ 
; ls =L || sea id) | column =E 


# A^AA^A^A^A^^ ^^ 


# The "sed 1d" in the pipe deletes the first line of output, 
#+ which would be "total IRI. 
#+ where "N" is the total number of files found by "ls -1". 


# The -t option to "column" pretty-prints a table. 


|Oobomobpmpmpmowmn 
OY O! 4 (0. I9. IP. O xo 00 -1 O OI 4S CQ) I9 S 


exit 0 


Column removal filter. This removes columns (characters) from a file and writes the file, lacking the 
range of specified columns, back to stdout. colrm 2 4 «filename removes the second 
through fourth characters from each line of the text file £ilename. 


If the file contains tabs or nonprintable characters, this may cause unpredictable 
behavior. In such cases, consider using expand and unexpand in a pipe preceding 


colrm. 


Line numbering filter: n1 filename lists filename to stdout, but inserts consecutive numbers 
at the beginning of each non-blank line. If filename omitted, operates on stdin. 


The output of nl is very similar to cat —b, since, by default nl does not list blank lines. 


Example 16-28. nl: A self-numbering script. 


#!/bin/bash 
# line-number.sh 


# This script echoes itself twice to stdout with its lines numbered. 


# 'cat -n' sees the above line as number 6. 


T 
2 
3 
4 
5 
6 4 'nl' sees this as line 4 since it does not number blank lines. 
7 
8 
9 nl `basename $0` 

0 

Y 


Senor echo a Wew, lerte tiny aie waia "xeu =m" 


12 
13 cat -n “basename $0" 

14 # The difference is that 'cat -n' numbers the blank lines. 
15 s; NEES eeu "sadi lom" wall als@ clo Se. 

16 

I7 esi 0) 

18 4 


pr 
Print formatting filter. This will paginate files (or st dout) into sections suitable for hard copy 
printing or viewing on screen. Various options permit row and column manipulation, joining lines, 
setting margins, numbering lines, adding page headers, and merging files, among other things. The pr 
command combines much of the functionality of nl, paste, fold, column, and expand. 


pr -o 5 --width-65 fileZZZ | more gives a nice paginated listing to screen of 
fileZZZ with margins set at 5 and 65. 


A particularly useful option is —d, forcing double-spacing (same effect as sed -G). 

gettext 
The GNU gettext package is a set of utilities for localizing and translating the text output of programs 
into foreign languages. While originally intended for C programs, it now supports quite a number of 
programming and scripting languages. 


The gettext program works on shell scripts. See the info page. 
msgfmt 
A program for generating binary message catalogs. It is used for localization. 


Icony 
A utility for converting file(s) to a different encoding (character set). Its chief use is for localization. 
1 # Convert a string from UTF-8 to UTF-16 and print to the BookList 
2 function write utf8 string { 
3 STRING-$1 
4 BOOKLIST-$2 
5 echo m "SSTRING" | iconv -f UTF8 -t UIF16 | \ 
6 cut o-b 3— | tr d \\n >> "SBOOKETZSTI" 
T o 
8 
9 # From Peter Knowles' "booklistgen.sh" script 
10 #+ for converting files to Sony Librie/PRS-50X format. 
11 #  (http://booklistgensh.peterknowles.com) 
recode 


Consider this a fancier version of iconv, above. This very versatile utility for converting a file to a 
different encoding scheme. Note that recode is not part of the standard Linux installation. 

TeX, gs 
TeX and Postscript are text markup languages used for preparing copy for printing or formatted 
video display. 


TeX is Donald Knuth's elaborate typsetting system. It is often convenient to write a shell script 
encapsulating all the options and arguments passed to one of these markup languages. 


Ghostscript (gs) is a GPL-ed Postscript interpreter. 

texexec 
Utility for processing TeX and pdf files. Found in /usr/bin on many Linux distros, it is actually a 
shell wrapper that calls Perl to invoke Tex. 


1 texexec --pdfarrang result-Concatenated.pdf *pdf 
2 


3 4 Concatenates all the pdf files in the current working directory 

4 #+ into the merged file, Concatenated.pdf 

5 # (The --pdfarrange option repaginates a pdf file. S also pdfcombine.) 

6 # The above command-line could be parameterized and put into a shell script. 


enscript 
Utility for converting plain text file to PostScript 


For example, enscript filename.txt -p filename.ps produces the PostScript output file 
filename.ps. 

groff, tbl, eqn 
Yet another text markup and display formatting language is groff. This is the enhanced GNU version 
of the venerable UNIX roff/troff display and typesetting package. Manpages use groff. 


The tbl table processing utility is considered part of groff, as its function is to convert table markup 
into groff commands. 


The eqn equation processing utility is likewise part of groff, and its function is to convert equation 
markup into groff commands. 


Example 16-29. manview: Viewing formatted manpages 


1 #!/bin/bash 

2 manview.sh: Formats the source of a man page for viewing. 
3 

4 This script is useful when writing man page source. 

5 It lets you look at the intermediate results on the fly 
6 #+ while working on it. 

7 

8 E_WRONGARGS=85 

9 

Jg a | —- YS” | 

11 then 

112 echo "Usage: 'basename $0! filename" 

1,5 exit $E WRONGARGS 

Jd a 

15 

16 

i] gror -Tasci wem Si | less 

JL) From the man page for groff. 

19 
20 
2 If the man page includes tables and/or equations, 
22 #+ then the above code will barf. 
23) The following line can handle such cases. 
24 
25 ejclol < WS | geca -—Tlarimil | gror —Wwileicdinill -me cy- Chan menn 
26 
27 Thanks, S.C. 
28 
AS) wap Se # See also the "maned.sh" script. 


See also Example A-39. 
lex, yacc 


The lex lexical analyzer produces programs for pattern matching. This has been replaced by the 
nonproprietary flex on Linux systems. 


The yacc utility creates a parser based on a set of specifications. This has been replaced by the 
nonproprietary bison on Linux systems. 


Notes 


[1] This is only true of the GNU version of tr, not the generic version often found on commercial UNIX 
systems. 
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16.5. File and Archiving Commands 


Archiving 


tar 


shar 


ar 


rpm 


The standard UNIX archiving utility. [1] Originally a Tape ARchiving program, it has developed into 
a general purpose package that can handle all manner of archiving with all types of destination 
devices, ranging from tape drives to regular files to even st dout (see Example 3-4). GNU tar has 
been patched to accept various compression filters, for example: tar czvf archive name.tar.gz *, 
which recursively archives and gzips all files in a directory tree except dotfiles in the current working 


directory (SPWD). [2] 


Some useful tar options: 


1. -c create (a new archive) 
2. -x extract (files from existing archive) 
3. - delete delete (files from existing archive) 


D This option will not work on magnetic tape devices. 


. —r append (files to existing archive) 

. —A append (tar files to existing archive) 

. —t list (contents of existing archive) 

. —u update archive 

. -d compare archive with specified filesystem 

after-date only process files with a date stamp after specified date 
. —z gzip the archive 


SOONDNSA 


pà 


(compress or uncompress, depending on whether combined with the -c or —x) option 
11. -j bzip2 the archive 
It may be difficult to recover data from a corrupted gzipped tar archive. When 
archiving important files, make multiple backups. 


Shell archiving utility. The text files in a shell archive are concatenated without compression, and the 
resultant archive is essentially a shell script, complete with #!/bin/sh header, containing all the 
necessary unarchiving commands, as well as the files themselves. Shar archives still show up in 
Usenet newsgroups, but otherwise shar has been replaced by tar/gzip. The unshar command 
unpacks shar archives. 


The mailshar command is a Bash script that uses shar to concatenate multiple files into a single one 
for e-mailing. This script supports compression and uuencoding. 


Creation and manipulation utility for archives, mainly used for binary object file libraries. 


The Red Hat Package Manager, or rpm utility provides a wrapper for source or binary archives. It 
includes commands for installing and checking the integrity of packages, among other things. 


A simple rpm -i package_name.rpm usually suffices to install a package, though there are many 
more options available. 


i) rpm -qf identifies which package a file originates from. 


bash$ rpm -qf /bin/ls 
corcur 189-5.2.1-3 


rpm -qa gives a complete list of all installed rpm packages on a given system. An 
rpm -qa package name lists only the package(s) corresponding to 
package name. 


— 


bash$ rpm -qa 
iecexollavere; — 1498199. «il, S1 
JEDET 7 «2r cdi cd) 
Gueeielildl9-2 47-12 
glos sik OO S2 , 7—1 
Golam -La Ss O= 1:10) 
ksymoops-2.4.1-1 
mktemp-1.5-11 
[oXexedl i3 der e (orba) 
WSL SSI e — ved l8) 9) 5» 9 3] 24 


bash$ rpm -qa docbook-utils 
elexeloxoxols- ttai 0) » (6. 9). 


bash$ rpm -qa docbook | grep docbook 
ieloxeloxoxolis —iele(o lS) 1L— exea —31. o (0) 30 
docbook-style-dsssl-1.064-3 

doche ok lieri .— sedi — 1. , (91:0) 
docbook-dtd40-sgml-1.0-11 

Gloxeloroxolis iet 1Lei-19xelie — (0) - (9 «917 

elexelexexollis ehe t 1L— exer 15 5 (0) 1:0 
eecloxoxolk wc. 1:5—(0),(5 5 9-2 


cpio 
This specialized archiving copy command (copy input and output) is rarely seen any more, having 
been supplanted by tar/gzip. It still has its uses, such as moving a directory tree. With an appropriate 
block size (for copying) specified, it can be appreciably faster than tar. 


Example 16-30. Using cpio to move a directory tree 


! /bin/bash 
Copying a directory tree using cpio. 


Advantages of using 'cpio': 
Speed of copying. It's faster than 'tar' with pipes. 
Well suited for copying special files (named pipes, etc.) 
4r chaw "Gp" mey Choke (GO. 


ARGS-2 
E BADARGS-65 


A cHem S ARGS E] 


echo "Usage: 'basename $0` source destination" 
exit $E BADARGS 


source="$1" 
destination-"$2" 


NPRPRPPRP PPP PY 
O io 0» -1 O Oi i (). NM P. O xo 0 -1 O O1 4 CQ Io E 


Zt 
22 FEET EEE EE HEHE HE EE HEH HE EERE EE E E E EE E E E EE E E E E EE EEE EE EE HE HE EH 
29 rinci US cepta || cblo uuo: | Sxcleisiic ainelic sl oua 


24 SOHO d QU Qu 

25 Read the 'find' and 'cpio' info pages to decipher these options. 
26 The above works only relative to $PWD (current directory) 

27 #+ full pathnames are specified. 

28 FEE TE HE TEIE HE HE E IE HE TE FE HE FE E FE HE FE FE HE FE HE FE E FE FE HE FE HE FE HE FE FE FE FE HE FE E FE E FE FE HE FE HE TE FE FE FE AE FE E TE FE E FE HE E E E E E E E E H 
29 

30 

Sl Exercise: 

32 4 ——————-- 

33 

34 Add code to check th sub Siteiewls:! (SP) wie ches Vrime || yeso" ose 
35 #+ and output appropriat rror messages if anything went wrong. 

36 

Si ese S 


rpm2cpio 
This command extracts a cpio archive from an rpm one. 


Example 16-31. Unpacking an rpm archive 


#!/bin/bash 
# de-rpm.sh: Unpack an 'rpm' archive 


$(1?"Usage: 'basename SO target-file") 
# Must specify 'rpm' archive name as an argument. 


TEMPFILE-$$.cpio # Tempfile with "unique" name. 
w SS ie procese ID of sesrXpt. 


rpm2cpio < $1 > STEMPFILE # Converts rpm archive into 
#+ cpio archive. 

CLO make-directories -F $TEMPFILE -i # Unpacks cpio archive. 

rm -f STEMPFILE # Deletes cpio archive. 


exse (0) 


# Exercise: 

# Add check for whether 1) "target-file" exists and 

#+ 2) it is an rpm archive. 

# Hint: Parse output of 'file' command. 


[— €x» We) (Gor a) cx, Wal dex Go [soy [9 €» Wer sr c] ny Gal des (es) Iss) [S 


[xy [e t 


pax 
The pax portable archive exchange toolkit facilitates periodic file backups and is designed to be 
cross-compatible between various flavors of UNIX. It was ported from BSD to Linux. 


pax -wf daily backup.pax -/linux-server/files 
Creates a tar archive of all files in the target directory. 
Note that the options to pax must be in the correct order 
+ pax -fw has an entirely different effect. 


pax -f daily backup.pax 
Lists the files in the archive. 


9 pax -rf daily backup.pax -/bsd-server/files 
10 Restores the backed-up files from the Linux machine 
11 #+ onto a BSD one. 


Note that pax handles many of the standard archiving and compression commands. 


Compression 


gzip 
The standard GNU/UNIX compression utility, replacing the inferior and proprietary compress. The 
corresponding decompression command is gunzip, which is the equivalent of gzip -d. 


$^; The -c option sends the output of gzip to stdout. This is useful when piping to 
other commands. 


The zcat filter decompresses a gzipped file to st dout, as possible input to a pipe or redirection. This 
is, in effect, a cat command that works on compressed files (including files processed with the older 
compress utility). The zcat command is equivalent to gzip -dc. 


On some commercial UNIX systems, zcat is a synonym for uncompress -c, and will 
not work on gzipped files. 
See also Example 7-7. 
bzip2 
An alternate compression utility, usually more efficient (but slower) than gzip, especially on large 
files. The corresponding decompression command is bunzip2. 


Similar to the zcat command, bzcat decompresses a bzipped2-ed file to stdout. 


cg) Newer versions of tar have been patched with bzip2 support. 


compress, uncompress 
This is an older, proprietary compression utility found in commercial UNIX distributions. The more 
efficient gzip has largely replaced it. Linux distributions generally include a compress workalike for 
compatibility, although gunzip can unarchive files treated with compress. 


į ) The znew command transforms compressed files into gzipped ones. 


sq 
Yet another compression (squeeze) utility, a filter that works only on sorted ASCII word lists. It uses 
the standard invocation syntax for a filter, sq < input-file > output-file. Fast, but not nearly as 
efficient as gzip. The corresponding uncompression filter is unsq, invoked like sq. 
į ) The output of sq may be piped to gzip for further compression. 
zip, unzip 


Cross-platform file archiving and compression utility compatible with DOS pkzip.exe. "Zipped" 
archives seem to be a more common medium of file exchange on the Internet than "tarballs." 

unarc, unarj, unrar 
These Linux utilities permit unpacking archives compressed with the DOS arc.exe, arj.exe, and 
rar.exe programs. 

Izma, unlzma, Izcat 
Highly efficient Lempel-Ziv-Markov compression. The syntax of /zma is similar to that of gzip. The 
7-zip Website has more information. 


File Information 


file 
A utility for identifying file types. The command file file-name will return a file specification 
for file-name, suchas ascii text or data. It references the magic numbers found in 
/usr/share/magic, /etc/magic, or /usr/lib/magic, depending on the Linux/UNIX 
distribution. 


The -£ option causes file to run in batch mode, to read from a designated file a list of filenames to 
analyze. The -z option, when used on a compressed target file, forces an attempt to analyze the 


uncompressed file type. 


bash$ file test.tar.gz 

test.tar.gz: gzip compressed data, deflated, 

last mockiitiece Sum Sep 16 JX3;s94551 200). ems Wins 
bash file -z test.tar.gz 

test.tar.gz: GNU tar archive (gzip compressed data, 
last mockiitiec: Sum See 16 J 98254551 2001, oss Wins) 


# Find sh and Bash scripts in a given directory: 


al 

2 

3 DIRECTORY-/usr/local/bin 
4 KEYWORD-Bourne 
5 
6 
y 
8 


# Bourne and Bourne-Again shell scripts 


file SDIRECTORY/* 


fgrep SKEYWORD 


9) a? (OXUNEJSXENE.E 
10 
Li qe (wee / Local / Torstrgw/ exon rel e Bourne-Again 
12 # /usr/local/bin/burnit: Bourne-Again 
13 # /usr/local/bin/cassette.sh: Bourne shell 
14 # /usr/local/bin/copy-cd: Bourne-Again 
15 # 


Example 16-32. Stripping comments from C program files 


!/bin/bash 
strip-comment.sh: Strips out the comments 
E NOARGS-0 

E ARGERROR-66 

E WRONG FILE TYPE-67 


xs [ 
then 
echo "Usag 
exit $E ARGE 
ftat 


$4 -eq "SE NOARGS" ] 


$0^ 


"basenam 
RROR 


# Test for correct file type. 

type- file $1 | awk '{ print $2, 
s "isle S1" echoes iile Eyo ae 
# Then awk removes the first field, the filenam 


S37 S4, SS PUT 


= 


C-program-file" >&2 # 


(os: cp xexy Cyl des (es) TA Sp We) (er cp cx Gal des (ee) JS» e 


# Then the result is fed into the variable 
correct type-"ASCII C program text" 


N 
© Xo 


if [ 
then 
echo 


Wise! IE Weropersicia ews! | 


NO NO PRO PO PN 
(Og: dE (G8) [eor pv 


echo 
exit 
Tal 


Xn 


E WRONG FILE TYPE 


NO NO 
- OY 


N 
[99] 


29 
SI) c5 icicle (uewasibske. Sel eripe: 
Sil 


deflated, 


shell script 
shell script 


Script text 
shell script 


(/* COMMENT */) 


Teyoe. @ 


exo, "8H SERIO works om © procram elas emily,” 


Error Message Lo 


xt executabl 
xt executabl 
cutabl 

xt executabl 


in a C program. 


stderr. 


22 
33 
34 
35) 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
33 
54 
55 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
pu 
72 
WS 
74 
T3 
76 
77] 
78 
HS 


sed ' 
/ ^N NS Il 
[NENG el 
' Si 

Easy to understand if you take several hours to learn sed fundamentals. 

Need to add one more line to the sed script to deal with 

+ case where line of code has a comment following it on same line. 

This is left as a non-trivial exercise. 

Also, the above code deletes non-comment lines with a "*/" 

+ not a desirable result. 
exit 0 
# 
# Code below this line will not execute because of ‘exit 0' above. 


# Stephane Chazelas suggests the following alternative: 


usage() { 
echo "Usage: 'basename $0° C-program-file" >&2 
exit 1 
} 
WEIRD-'echo -n -e 'N377'^ # or WEIRD=$'\377' 
[[ $# -eq 1 ]] || usage 
Cage "iade WSU 3m 
*WC program text"*) sed -e "s%/\*SS{WEIRD}%gq;S%\*/SS{WEIRD} 3g" "Si" N 
| wz UNSTTWGU UNGNSZTS YÀ 
| sed -ne 'p;n' \ 
| sexe Sel UNUS |) dere VSI UNIES 
*) usage; ; 
esac 
This is still fooled by things like: 
oiire (f eg 
or 
/* /* buggy embedded comment */ 
To handle all special cases (comments in strings, comments in string 
+ wüsexe there is a V. WU soch, 
+ the only way is to write a C parser (using lex or yacc perhaps?) . 
exit 0 


which 


which command gives the full path to "command." This is useful for finding out whether a particular 
command or utility is installed on the system. 


$bash which rm 


/usr/bin/rm 


For an interesting use of this command, see Example 35-14. 


whereis 


Similar to which, above, whereis command gives the full path to "command," but also to its 
manpage. 


$bash whereis rm 


whatis 


vdir 


rm: /bin/rm /usr/share/man/manl/rm.1.bz2 


whatis command looks up "command" in the what is database. This is useful for identifying system 
commands and important configuration files. Consider it a simplified man command. 


$bash whatis whatis 


whatis (1) - search the whatis database for complete words 


Example 16-33. Exploring /usr/X11R6/bin 


#!/bin/bash 


# What are all those mysterious binaries in /usr/X11R6/bin? 


DIRECTORY="/usr/X11R6/bin" 
7 way also "faim", "U/wugr/lm. V"/me/local/sum". GEG. 


al 
2 
3 
4 
5 
6 
F 
8 for file in $DIRECTORY/* 
$ elo 
10 whatis 'basename $file' # Echoes info about the binary. 
i 
2 
E 
4 
5 
6 
7 
8 


done 
exit 0) 


# You may wish to redirect output of this script, like so: 
4 ./what.sh >>whatis.db 

# or view it a page at a time on stdout, 

# ./what.sh | less 


See also Example 11-3. 


Show a detailed directory listing. The effect is similar to Is -Ib. 


This is one of the GNU fileutils. 


bash$ vdir 

iol JL) 

CUENTE iL oze DOZO 20/9) awl 1S 22704 catal rolg 

CEW e eo iL lezo bozo 4602 May 25 13:58 datal.xrolo.bak 
STA IS 1 bozo bozo 877 Dec 17 2000 employment .xrolo 
bash ls -1 

coral 31) 

SEW eS = 1 bozo bozo 20/54 wii Ie 22 eal earal rola 
cud iL bozo bozo 4602 May 25 13:58 datal.xrolo.bak 
Saye ea DO ZO O20 877 Dec 17 2000 employment .xrolo 


locate, slocate 


The locate command searches for files using a database stored for just that purpose. The slocate 
command is the secure version of locate (which may be aliased to slocate). 


Sbash locate hickson 


/usr/lib/xephem/catalogs/hickson.edb 


getfacl, setfacl 


These commands retrieve or set the file access control list -- the owner, group, and file permissions. 


bash$ getfacl * 


# file: 


testl.txt 


# owner: bozo 

# group: bozgrp 
user: :rw- 
group: :rw- 
other: G- 


3 fes 


test2.txt 


# owner: bozo 

# group: bozgrp 
user: :rw- 
geOUPE ENE 
other: r- 


bash$ setfacl -m u:bozo:rw yearly budget.csv 
bash$ getfacl yearly budget.csv 


# file: 


yearly. budget.csv 


# owner: accountant 
# group: budgetgrp 
user::rw- 
user:bozo:rw- 


usertraccountent:rw- 
J (ONUS B B TEN 
mask::rw- 

(Quel (ese E — 


readlink 
Disclose the file that a symbolic link points to. 


strings 


bash$ readlink /usr/bin/awk 
sof a co d OLD Gayle 


Use the strings command to find printable strings in a binary or data file. It will list sequences of 
printable characters found in the target file. This might be handy for a quick 'n dirty examination of a 


core dump 


or for looking at an unknown graphic image file(strings image-file | more 


might show something like JFIF, which would identify the file as a jpeg graphic). In a script, you 
would probably parse the output of strings with grep or sed. See Example 11-7 and Example 11-9. 


Example 16-34. An "improved" strings command 


1 #!/bin/bash 

2 wstrings.sh: "word-strings" (enhanced "strings" command) 
3 

4 This script filters the output of "strings" by checking it 
5 #+ against a standard word list file. 

6 This effectively eliminates gibberish and noise, 

7 #+ and outputs only recognized words. 

8 

9 

LO Standard Check for Script Argument (s) 

11 ARGS=1 

12 E_BADARGS=85 

13 E NOFILE-86 

14 


15 if [ $$ -ne SARGS | 
16 then 
17 echo "Usage: 'basename $0! filename" 


ES exit SE_BADARGS 
I9 itu 
20 
Pik 3t qp Wes US a] # Check if file exists. 
22 then 
2.3 echo "File N"$1N" does not exist." 
24 exit $E NOFILE 
25 it 
26 # 
227) 
28 
29 MINSTRLEN=3 # Minimum string length. 
30 WORDFILE-/usr/share/dict/linux.words # Dictionary file. 
31 # May specify a different word list file 
32 #+ of one-word-per-line format. 
33 4$ For example, the "yawl" word-list package, 
34 4 http://bash.neuralshortcircuit.com/yawl-0.3.2.tar.gz 
33 
36 
ey lesee erea e "US |] aie A Gr | aie Y [espacasy]! 5 N 
Se ue CS "[sedpeeas]" 5X | ra= S NANT A || wie A ov v 
3g 
40 Translate output of 'strings' command with multiple passes of 'tr'. 
41 "tr A-Z a-z" converts to lowercase. 
42 "tr '[:space:]'" converts whitespace characters to Z's. 
43 Wor (ee ‘|| galloinesg ||! 4" «xot mom—Ellomalositne Cliguecetecs to Z5. 
44 #+ and squeezes multiple consecutive Z's. 
45 Hee esp UXdUTS-WS7TU A Converts all characters past "v" to dg 
46 #+ and squeezes multiple consecutive Z's, 
47 #+ which gets rid of all the weird characters that the previous 
48 #+ translation failed to deal with. 
49 Finally, "tr Z ' '" converts all those Z's to whitespace, 
50 #+ which will be seen as word separators in the loop below. 
5/1 
52 kkkxkxkxkkxkxkxkxkxkkxkkxkxkxkkkxkxkxkxkxkkxkxkxkxkxkxkxkxkxkxkxkxkkxkxkxkxkxkkxkxkxkxkxkxkxkxkxkxkxkxkkkxkxkkxkxx*k 
53 Note the technique of feeding the output of 'tr' back to itself, 
54 #+ but with different arguments and/or options on each pass. 
595) Ck ck ck ck ck Ck ck ck Ck ck CC ck Ck ck kk ck Ck ck ck ck ck ck ck ck Ck ck ck ck ck ck ck ck Ck ck ck ck ck Ck ck ck ck ck kk ck kk ko ko ck ko Sk ck Mk Sk M ko ko kx ko 
56 
57 
58 for word in $wlist Important: 
59 $wlist must not be quoted here. 
60 "Swlist" does not work. 
61 Why not? 
62 do 
63 
64 strlen=S {#word} String length. 
65 aie (| VSeiewikemY ile VSRMORHNISUDEUHDENDU T Skip over short strings. 
66 then 
67 continue 
68 ipa 
69 
70 grep -Fw Sword "SWORDFILE" # Match whole words only. 
71 d te # "Fixed strings" and 
72 #+ "whole words" options. 
33 
74 done 
TS 
76 
Ti Se Sg 
Comparison 


diff, patch 


diff: flexible file comparison utility. It compares the target files line-by-line sequentially. In some 
applications, such as comparing word dictionaries, it may be helpful to filter the files through sort and 
uniq before piping them to diff. diff file-1 file-2 outputs the lines in the files that differ, 
with carets showing which file each particular line belongs to. 


The --side-by-side option to diff outputs each compared file, line by line, in separate columns, 
with non-matching lines marked. The -c and -u options likewise make the output of the command 
easier to interpret. 


There are available various fancy frontends for diff, such as sdiff, wdiff, xdiff, and mgdiff. 


į ) The diff command returns an exit status of 0 if the compared files are identical, and 1 
if they differ. This permits use of diff in a test construct within a shell script (see 
below). 
A common use for diff is generating difference files to be used with patch The -e option outputs 
files suitable for ed or ex scripts. 


patch: flexible versioning utility. Given a difference file generated by diff, patch can upgrade a 
previous version of a package to a newer version. It is much more convenient to distribute a relatively 
small "diff" file than the entire body of a newly revised package. Kernel "patches" have become the 
preferred method of distributing the frequent releases of the Linux kernel. 


1 patch -pl «patch-file 

2 # Takes all the changes listed in 'patch-file' 

3 4 and applies them to the files referenced therein. 
4 # This upgrades to a newer version of the package. 


Patching the kernel: 
i egli usse 
2 puo -ec peesox.m | pacen uU 
3 4 Upgrading kernel source using 'patch'. 
4 4 From the Linux kernel docs "README", 
5 4 by anonymous author (Alan Cox?). 


$^; The diff command can also recursively compare directories (for the filenames 
present). 


bash$ diff -r -«/notesl ~/notes2 
Only in /home/bozo/notesl: file02 
Only in /home/bozo/notesl: file03 
Only in /home/bozo/notes2: file04 


Use zdiff to compare gzipped files. 


Use diffstat to create a histogram (point-distribution graph) of output from diff. 
diff3, merge 
An extended version of diff that compares three files at a time. This command returns an exit value of 
0 upon successful execution, but unfortunately this gives no information about the results of the 
comparison. 


bash$ diff3 file-1 file-2 file-3 


ibe ale 
Masks sls} line i oie “Wien eal 
2r Te 


sdiff 


cmp 


comm 


Wane als) Jdbsheve: JL «uir Warn ike—2 
33 Ike 
Maske: as: ne db ee aei 


The merge (3-way file merge) command is an interesting adjunct to diff3. Its syntax is merge 
Mergefile filel file2. The result is to output to Mergefile the changes that lead from 
filel to file2. Consider this command a stripped-down version of patch. 


Compare and/or edit two files in order to merge them into an output file. Because of its interactive 
nature, this command would find little use in a script. 


The emp command is a simpler version of diff, above. Whereas diff reports the differences between 
two files, cmp merely shows at what point they differ. 


2^; Like diff, cmp returns an exit status of 0 if the compared files are identical, and 1 if 
they differ. This permits use in a test construct within a shell script. 


Example 16-35. Using cmp to compare two files within a script. 


!/bin/bash 


ARGS-2 # Two args to script expected. 
E BADARGS-65 
| UNREADABLE-66 


if [ $$ -ne "SARGS" ] 


echo "Usage: `basename $0^ filel file2" 
exit $E BADARGS 


fe quip 3 xe. OenE p sic is uSAZU 

then 

echo "Both files to be compared must exist and be readable." 
exit $E UNREADABLE 


ita 


cmp $1 $2 &» /dev/null 4 /dev/null buries the output of the "cmp" command. 


Mop pop pop PPP PY 
O (o 0» 1 O Oi i (). NM IB. O xo 0 -1 O O1 4 QI 
Fh 
E 


# emis -e Sil S2 has game recule (Ue silent icilatey ice Hemo) 
21 # Thank you Anders Gustavsson for pointing this out. 
22 t 
DON ENUASNS ONEWOZICS RENE RA CH feci UP MET SH pbi Sil $2 ws gni 
24 
25. ave || Sg eee 1 # Test exit status of "cmp" command. 
26 then 
BY Geno Winsi the NUISILNU ates aielenniesieeul due ial: WS Yu 
28 else 
29 excl; MPLS WSL obim enas areont del NUN. 
SO sea, 
Sil 
S2. exu O) 


į ) Use zemp on gzipped files. 


Versatile file comparison utility. The files must be sorted for this to be useful. 


comm -options first-file second-file 


comm file-1 file-2 outputs three columns: 


9 column 1 = lines unique to file-1 
9 column 2 = lines unique to file-2 
© column 3 = lines common to both. 
The options allow suppressing output of one or more columns. 


9 —1 suppresses column 1 

© -2 suppresses column 2 

© -3 suppresses column 3 

© -12 suppresses both columns 1 and 2, etc. 
This command is useful for comparing "dictionaries" or word lists -- sorted text files with one word 
per line. 


Utilities 


basename 
Strips the path information from a file name, printing only the file name. The construction 
basename $0 lets the script know its name, that is, the name it was invoked by. This can be used 
for "usage" messages if, for example a script is called with missing arguments: 


1 echo "Usage: basename $0' argl arg2 ... argn" 
dirname 
Strips the basename from a filename, printing only the path information. 


8") basename and dirname can operate on any arbitrary string. The argument does not 
need to refer to an existing file, or even be a filename for that matter (see Example 
A-T). 


Example 16-36. basename and dirname 


1 #!/bin/bash 

2 

3 a-/home/bozo/daily-journal.txt 

4 

5 echo "Basename of /home/bozo/daily-journal.txt = basename $a'" 

6 echo "Dirname of /home/bozo/daily-journal.txt = “dirname $a'" 

7 echo 

8 echo "My own home is 'basename -/'." # ^basename ~`ò also works. 
9 echo "The home of my home is “dirname -/^." # “dirname ~*~ also works. 
10 

iit exw (0) 


split, csplit 
These are utilities for splitting a file into smaller chunks. Their usual use is for splitting up large files 
in order to back them up on floppies or preparatory to e-mailing or uploading them. 


The csplit command splits a file according to context, the split occuring where patterns are matched. 


Example 16-37. A script that copies itself in sections 


1 #!/bin/bash 
2 # splitcopy.sh 
3 


i? A eermupu imet solitis steel mto clus, 
#+ then reassembles the chunks into an exact copy 
#+ of the original script. 


CHUNKSIZE-4 Ww Size Qu ix cl Oi eps tiles. 
OUTPREFIX-xx # csplit prefixes, by default, 
#+ files with "xx" 


csplit "$0" "SCHUNKSIZE 


Some comment lines for padding 
Line 15 
Line 16 
Line 17 
Line 18 
Line 19 
Line 20 


+ + cHe ce che GbR c 


cat "SOUTPREFIX"* > "$0.copy" # Concatenate the chunks. 
rm "SOUTPREFIX"* # Get rid of the chunks. 


Re [m3 [RA (RS [S qp ee eee rg 
(os) [ey [=> c9) (Wer (Ger p ep) (Gn dex (63. [5 [5 eS) (er (oe c] rexy (ab ES 


N N 
ow 


exp S? 


Encoding and Encryption 


sum, cksum, md5sum, shalsum 
These are utilities for generating checksums. A checksum is a number [3] mathematically calculated 
from the contents of a file, for the purpose of checking its integrity. A script might refer to a list of 
checksums for security purposes, such as ensuring that the contents of key system files have not been 
altered or corrupted. For security applications, use the md5sum (message digest 5 checksum) 
command, or better yet, the newer shalsum (Secure Hash Algorithm). [4] 


bash$ cksum /boot/vmlinuz 
1670054224 804083 /boot/vmlinuz 


bash$ echo -n "Top Secret" | cksum 
3391003827 10 


bash$ md5sum /boot/vmlinuz 
0f43eccea8f09e0a0b2b5cfldcf333ba  /boot/vmlinuz 


bash$ echo -n "Top Secret" | md5sum 
8babc97a6f62a4649716f4df8d61728f - 


£^) The cksum command shows the size, in bytes, of its target, whether file or stdout. 


The md5sum and shalsum commands display a dash when they receive their input 
from stdout. 


Example 16-38. Checking file integrity 


!/bin/bash 
file-integrity.sh: Checking whether files in a given directory 
have been tampered with. 


E DIR NOMATCH-70 
E BAD DBFILE-71 


fp) (Gil dex (es) [ey [5 


(oe) Sa) (ox (ub dE ©) IS) [= €» ke (eb cl 


68 


dbfile-File record.md5 


# Filename for storing records 


set_up_database () 


{ 


colo Scueeo or y SEM 
# Write directory name to first line of file. 
md5sum "S$directory"/* >> "$dbfile" 

# Append md5 checksums and filenames. 


check database () 


{ 


local n=0 
local filename 
local checksum 


(database file). 


# # 

# This file check should be unnecessary, 

#+ but better safe than sorry. 

zi (p 2 ae VUSeleiile i] 

then 
echo "Unable to read checksum database file!" 
exit $E BAD DBFILE 

E 

# # 


while read record[n] 
do 


directory checked-"$(record[0])" 
if [ "Sdirectory checked" != "$directory" ] 


then 


echo "Directories do not match up!" 
# Tried to use file for a different directory. 


exit $E DIR NOMATCH 
HEIN 


iie qp Ve gw oQ | # Not directory name. 


then 


filename[n]-2$( echo ${record[$n]} | 
#  md5sum writes records backwards, 
#+ checksum first, then filename. 

checksum[n]=$( md5sum "S${filename[n]}" ) 


aie "(4 qao $2 T" j 


zs [p "Sdseeoswelmlj« e vsSTcemeelswsn[m] A 


then 


echo "${filename[n]} unchanged." 


elif [ "'basename ${filenam 


Tiel v 


!- "Sdbfile" ] 


# Skip over checksum database file, 
#+ as it will change with each invocation of script. 


3c cnis SSIealjore Om (END). 


Eeneiige EENES 
then 
echo "S{filename[n] } 


iE aL 


This unfortunately means that when running 


tampering with the 


CH 


+ checksum database file will not be detected. 


ECKSUM 


ERROR!" 


File has been changed since last checked. 


13 E, 

74 

US 

76 

3 3 leg Unm 

78 done «Solo le" # Read from checksum database file. 
po 

80 ) 

81 

82 # # 

83 # main () 

84 

e sine [pw mU T 

86 then 

87 directory="SPWD" # If not specified, 

88 else #+ use current working directory. 

89 directory="$1" 

QO sea 

91 

92 clear # Clear screen. 

93 echo " Running file integrity check on S$directory" 

94 echo 

95 

96 # # 
97 if [ ! -r "Sdbfile" ] # Need to create database file? 

98 then 

99 echo "Setting up database file, \""Sdirectory"/"Sdbfile"\"."; echo 
100 set up database 
1O Ea 
102 # # 
103 
104 check_database # Do the actual work. 
105 
106 echo 
107 
108 # You may wish to redirect the stdout of this script to a file, 
109 #+ especially if the directory checked has many files in it. 
110 
iii esate (0) 
I2 
113 # For a much more thorough file integrity check, 
114 #+ consider the "Tripwire" package, 
115 #+ http://sourceforge.net/projects/tripwire/. 
LLG 


Also see Example A-19, Example 35-14, and Example 10-2 for creative uses of the md5sum 
command. 


H) There have been reports that the 128-bit md5sum can be cracked, so the more secure 
160-bit shalsum is a welcome new addition to the checksum toolkit. 


bash$ md5sum testfile 
e181e2c8720c60522c4c4c981108e367 testfil 


bash$ shalsum testfile 
5d7425a9c08a66c3177£1e31286fa40986ffc996 testfile 


Security consultants have demonstrated that even shalsum can be compromised. Fortunately, newer 
Linux distros include longer bit-length sha224sum, sha256sum, sha384sum, and sha512sum 
commands. 

uuencode 


This utility encodes binary files (images, sound files, compressed files, etc.) into ASCII characters, 
making them suitable for transmission in the body of an e-mail message or in a newsgroup posting. 
This is especially useful where MIME (multimedia) encoding is not available. 


uudecode 


This reverses the encoding, decoding uuencoded files back into the original binaries. 


Example 16-39. Uudecoding encoded files 


1 #!/bin/bash 
2 4 Uudecodes all uuencoded files in current working directory. 
3 
4 lines-35 # Allow 35 lines for the header (very generous). 
5 
© for Mile am 55 # Test all the files in SPWD. 
7 do 
8 searchl= head -n Slines $File | grep begin | wc -w` 
9 search2=`tail -n $lines $File | grep end | wc -w` 
LO # Uuencoded files have a "begin" near the beginning, 
JL #+ and an "end" near th nd. 
12 s | USseeucelaill’Y -zr Q | 
1.3 then 
14 sb [p USeeesaclm2" gue O T 
LS) then 
16 echo "uudecoding - $File -" 
13 uudecode $File 
18 aal, 
Wg) ia 
20 done 
AL 
22 Note that running this script upon itself fools it 
23 #+ into thinking it is a uuencoded file, 
24 #+ because it contains both "begin" and "end". 
25 
26 Exercise: 
Eq «uh ——————- 
28 Modify this script to check each file for a newsgroup header, 
29 #+ and skip to next if not found. 
30 
31 exit 0 


The fold -s command may be useful (possibly in a pipe) to process long uudecoded 


text messages downloaded from Usenet newsgroups. 
mimencode, mmencode 
The mimencode and mmencode commands process multimedia-encoded e-mail attachments. 
Although mail user agents (such as pine or kmail) normally handle this automatically, these particular 
utilities permit manipulating such attachments manually from the command-line or in batch 
processing mode by means of a shell script. 


crypt 


At one time, this was the standard UNIX file encryption utility. [5] Politically-motivated government 
regulations prohibiting the export of encryption software resulted in the disappearance of crypt from 
much of the UNIX world, and it is still missing from most Linux distributions. Fortunately, 
programmers have come up with a number of decent alternatives to it, among them the author's very 
own cruft (see Example A-4). 


openssl 


This is an Open Source implementation of Secure Sockets Layer encryption. 


1 # To encrypt a file: 
Z openssl aes-i28-eco -salt =m file txc -owt file- encryptecl \ 


# To decrypt the resulting tarball: 

openssl des3 -d -salt -in "Sencrfile" -pass pass:"Spassword" | 
14 tar RENI 

15 # Decrypts and unpacks into current working directory. 


Of course, openssl has many other uses, such as obtaining signed certificates for Web sites. See the 


3 -pass pass:my password 

4 inei roue OSL User-selected password. 

5 aes-128-ecb is the encryption method chosen. 

6 

7 To decrypt an openssl-encrypted fil 

8 openssl aes-128-ecb -d -salt -in file.encrypted -out file.txt \ 
9 -pass pass:my password 

10 O EE User-selected password. 

Piping openssl to/from tar makes it possible to encrypt an entire directory tree. 

il To encrypt a directory: 

2 

3 sourcedir="/home/bozo/testfiles" 

4 encrfile-"encr-dir.tar.gz" 

5 password=my_secret_password 

6 

J| tar Quei = USsouieesclaiic 

8 openssl des3 -salt -out "Sencrfile" -pass pass:"$password" 
9 d Qiu Uses des3 encryption. 
10 # Writes encrypted file "encr-dir.tar.gz" in current working directory. 
idt 
12 
13 


info page. 
shred 
Securely erase a file by overwriting it multiple times with random bit patterns before deleting it. This 
command has the same effect as Example 16-60, but does it in a more thorough and elegant manner. 
This is one of the GNU fileutils. 
Advanced forensic technology may still be able to recover the contents of a file, even 
after application of shred. 
Miscellaneous 
mktemp 


Create a temporary file [6] with a "unique" filename. When invoked from the command-line without 
additional arguments, it creates a zero-length file in the / tmp directory. 


bash$ mktemp 
/tmp/tmp.zzsvql3154 


PREF IX=filename 
tempfile-'mktemp S$PREFIX.XXXXXX' 
^^^^^^ Need at least 6 placeholders 
t in the filename templat 
If no filename template supplied, 
+ "tmp.XXXXXXXXXX" is the default. 


cho "tempfile name = $tempfile" 
tempfile name = filename.QA2ZpY 
or something similar... 


Creates a file of that name in the current working directory 
* with 600 file permissions. 
A "umask 177" is therefore unnecessary, 
+ but it's good programming practice anyhow. 


|ímb|bombpmpmonmn 
O40 NP. O10 -J1o0 0 4&0 NP|p 


make 


Utility for building and compiling binary packages. This can also be used for any set of operations 
triggered by incremental changes in source files. 


The make command checks a Makefile, a list of file dependencies and operations to be carried out. 


The make utility is, in effect, a powerful scripting language similar in many ways to Bash, but with 
the capability of recognizing dependencies. For in-depth coverage of this useful tool set, see the GNU 
software documentation site. 

install 
Special purpose file copying command, similar to cp, but capable of setting permissions and attributes 
of the copied files. This command seems tailormade for installing software packages, and as such it 
shows up frequently in Makefiles (inthe make install : section). It could likewise prove 
useful in installation scripts. 

dos2unix 
This utility, written by Benjamin Lin and collaborators, converts DOS-formatted text files (lines 
terminated by CR-LF) to UNIX format (lines terminated by LF only), and vice-versa. 

ptx 
The ptx [targetfile] command outputs a permuted index (cross-reference list) of the targetfile. This 
may be further filtered and formatted in a pipe, if necessary. 

more, less 
Pagers that display a text file or stream to stdout, one screenful at a time. These may be used to 
filter the output of stdout ... or of a script. 


An interesting application of more is to "test drive" a command sequence, to forestall potentially 
unpleasant consequences. 


i is /remellbozo || ank orini Sam sree Y Siu || morg 

2 # (SOUS 

B 

4 # Testing the effect of the following (disastrous) command-line: 
5 d ls /homeloczo || awk printe “iam —3á US GIPS | sih 

6 # Hand off to the shell to execute . . . io 


The /ess pager has the interesting property of doing a formatted display of man page source. See 
Example A-39. 


Notes 


[1] An archive, in the sense discussed here, is simply a set of related files stored in a single location. 


[2] Atar czvf ArchiveName.tar.gz *willinclude dotfiles in subdirectories below the current 
working directory. This is an undocumented GNU tar "feature." 


[3] The checksum may be expressed as a hexadecimal number, or to some other base. 
[4] For even better security, use the sha256sum, sha512, and shalpass commands. 
[5] 


This is a symmetric block cipher, used to encrypt files on a single system or local network, as opposed 
to the public key cipher class, of which pgp is a well-known example. 


[6] Creates a temporary directory when invoked with the -d option. 
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16.6. Communications Commands 


Certain of the following commands find use in network data transfer and analysis, as well as in chasing 
spammers. 


Information and Statistics 


host 


ipcalc 


Searches for information about an Internet host by name or IP address, using DNS. 


bash$ host surfacemail.com 
surfacemail.com. has address 202.92.42.236 


Displays IP information for a host. With the -h option, ipcale does a reverse DNS lookup, finding the 
name of the host (server) from the IP address. 


bash$ ipcalc -h 202.92.42.236 
HOSTNAME-surfacemail.com 


nslookup 


Do an Internet "name server lookup" on a host by IP address. This is essentially equivalent to ipcalc 
-h or dig -x . The command may be run either interactively or noninteractively, i.e., from within a 
script. 


The nslookup command has allegedly been "deprecated," but it is still useful. 


bash$ nslookup -sil 66.97.104.180 
nslookup kuhleersparnis.ch 
Server: 13552116551 9742 
Address: 135), 116,137 BSS 


Non-authoritative answer: 
Name: kuhleersparnis.ch 


Domain Information Groper. Similar to nslookup, dig does an Internet name server lookup on a host. 
May be run from the command-line or from within a script. 


Some interesting options to dig are +t ime=N for setting a query timeout to N seconds, +tnofail for 
continuing to query servers until a reply is received, and -x for doing a reverse address lookup. 


Compare the output of dig -x with ipcalc -h and nslookup. 


bash$ dig -x 81.9.6.2 
;; Got answer: 
;; —>>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 11649 
pg lages eye zo wap OURS I, ANSHIR: ©, AUTKORLEYs din ADDLETONALS (0) 


7) QUESTION SECTION: 
62.6.9 ,81 , din—aclclc arpar IN PTR 


PS AUTHORITY SECTION: 
(549.4 (1L; 303m—melehe » area. 3600 IN SOA ns.eltel.net. noc.eltel.net. 
2002031705 900 600 86400 3600 


;; Query time: 537 msec 
pp GiewaSS 195.116.197.99]59(195511625197532) 


itd 


TE 


WHEN: Wed Jun 26 08:35:24 2002 
MSG: SE EVER S 


Example 16-40. Finding out where to report a spammer 


nNerrrrrrEBEEH 
O (o 00 -1 O Qi SS M P. O xo 0 a O S NS S 


[SO RUDSO SETS MU ESO SO Soy TSS) 
=] (ex, (np dE. (93 |») [5 


CO CO CO CO CO CO CO CO CO CO DO DN 
WOOL (os) S| yy Gr PS ey IS) [5 em) We (99) 


op 
i- €» 


BoP gam am 
Ow WN 


!/bin/bash 
Spam-lookup.sh: Look up abuse contact to report a spammer. 
Thanks, Michael Zick. 


Check for command-line arg. 

ARGCOUNT-1 

E WRONGARGS-65 

if [ $4 -ne "SARGCOUNT" ] 

then 
echo "Usage: 'basename $0' domain-name" 
exit $E WRONGARGS 

ial 


Chie; sho ES conmisaetsimaluscmiuci M el ale) =i IESE 
Also try: 
dig +tnssearch $1 
Tries to find "authoritative name servers" and display SOA records. 


The following also works: 
whois -h whois.abuse.net $1 
^^ RAKAKRAKRAKAKAAKRARA Specify HOST.. 
Can even lookup multiple spammers with this, i.e." 
whois -h whois.abuse.net $spamdomainl $spamdomain2 


Exercise: 

Expand the functionality of this script 

tr SO) char ie etrtonerkbeeiliily Goines 2 JeXXEdLitdL CERE om 
+ to the responsible ISP's contact address(es). 
Hint: use the "mail" command. 


exu S 


spam-lookup.sh chinatietong.com 
A known spam domain. 


"crnet_mgr@chinatietong.com" 
"crnet_tec@chinatietong.com" 
"postmaster@chinatietong.com" 


For a more elaborate version of this script, 
+ see the SpamViz home page, http://www.spamviz.net/index.html. 


Example 16-41. Analyzing a spam domain 


(ox (On de €» Ps) [5 


#! /bin/bash 
# is-spammer.sh: Identifying spam domains 


s ordi sbm-epewwme, w 1.4 2004/09/31 392379252 wwe lego $ 
# Above line is RCS ID info. 
# 


7 This is a simplified version of the "is spammer.bash 
8 #+ script in the Contributed Scripts appendix. 
9 
10 is-spammer <domain.name> 
dL Ti. 
AZ, Uses an external program: 'dig' 
113 Tested with version: 9.2.4rc5 
14 
115 Uses functions. 
16 Uses IFS to parse strings by assignment into arrays. 
Ay And even does something useful: checks e-mail blacklists. 
18 
IS) Use the domain.name (s) from the text body: 
20 http://www.good stuff.spammer.biz/just ignore everything else 
Za TESTS ES ISIN OL 
22 Or the domain.name(s) from any e-mail address: 
d Really Good OfferGspammer.biz 
24 
25 as the only argument to this script. 
265 (PS: have your Inet connection running) 
27 
28 So, to invoke this script in the above two instances: 
2 is-spammer.sh spammer.biz 
30 
Sil 
32 Whitespace == :Space:Tab:Line Feed:Carriage Return: 
33 WS PRESS A20 DINERS OAM E NOD 
34 
35 No Whitespace == Line Feed:Carriage Return 
36 No_WSP=$'\x0A'$'\x0D' 
37 
38 Field separator for dotted decimal ip addresses 
39 ADR IFS-$(No WSP)'.' 
40 
41 Get the dns text resource record. 
42 get txt «error code» «list query» 
AS} eyes esac) | 
44 
45 # Parse $1 by assignment at the dots. 
46 local -a dns 
47 IFS=SADR_IFS 
48 dns=( $1 ) 
49 IFS=SWSP_IFS 
50 ie qp Vedeegng])" == X299 | 
Sik then 
52 # See if there is a reason. 
59 echo Scie rehort SZ =E TAE) 
54 fS 
55 j 
56 


57 4 Get the dns address resource record. 
58 4$ chk adr «rev dns» «list server» 
59 Gue admo 


60 local reply 

61 local server 

62 local reason 

63 

64 server=S{1}${2} 

65 reply-$( dig +short ${server} ) 
66 

67 # If reply might be an error code 
68 if [ S{#reply} -gt 6 ] 

69 then 

70 reason-$(get txt S${reply} $(server) ) 
Fal reason=${reason:-${reply}} 


12 ia 


echo $(reason:-' not blacklisted.') 


# Need to get the IP address from the name. 


cho 'Get address of: '$1 

ip_adr=$ (dig +short $1) 
dns reply-$(ip adr:-' no answer '} 
Found address: '$(dns reply) 


echo ' 


# A valid reply is at least 4 digits plus 3 dots. 
if [ ${#ip_adr} -gt 6 ] 


then 


echo 
declare query 


# Parse by assignment at the dots. 
declare -a dns 

IFS=SADR_IFS 

dns-( $(ip adr) ) 

IFS=SWSP_IFS 


# Reorder octets into dns query order. 


we (ola $ fehavs: Si] jj "9 » VS dieu [2p Ys POS eae 1] jr Va wedekms]9] jw» s 


# See: 
ec 
ec 


# See: 
ec 
ec 


# See: 
ec 
ec 


http://www.spamhaus.org (Conservative, well maintained) 


NO 
NO 


NO 
NO 


NO 
NO 


-n 'spamhaus.org says: ' 
S(chk_adr ${rev_dns} 'sbl-xbl.spamhaus.org') 


http://ordb.org (Open mail relays) 


=a d Qyeiclooiaey Cayes | 
S(chk_adr ${rev_dns} 'relays.ordb.org') 


http://www.spamcop.net/ (You can report spammers here) 


-im | BoEWMIeCIO ines Bayes | 
S(chk_adr ${rev_dns} 'bl.spamcop.net') 


# # # other blacklist operations # # # 


# See: 
ec 
ec 


# See: 
ec 
ec 
ec 
ec 


ec 
ec 


ec 
ec 


else 
ec 
ec 
agal 


Sae dO) 


http://cbl.abuseat.org. 


no 
no 


no 
NO 
NO 
NO 


no 
no 


no 


no 


no 
no 


-n ' abuseat.org says: ' 
$(chk adr S$(rev dns) 'cbl.abuseat.org') 


http://dsbl.org/usage (Various mail relays) 


'Distributed Server Listings' 
=m | Lisi Css owe saves ” 
S(chk adr S{rev_dns} "list -dsbl. org") 


ig multihop.dsbl.org says: ' 
S(chk_adr ${rev_dns} 'multihop.dsbl.org') 


=n YUINCGOMELMeC Coil ome Saves " 
S(chk_adr ${rev_dns} 'unconfirmed.dsbl.org') 


"Could not use that address.' 


# Exercises: 


# 1) Check arguments to script, 
# and exit with appropriat rror message if necessary. 


139 

14) p 2) Claeele is gelo: ere dumwiüocelb3u O1 Serie, 

141 # and exit with appropriat rror message if necessary. 

142 

143 4 3) Substitute generic variables for "hard-coded" BHL domains. 
144 

145 4 4) Set a time-out for the script using the "+time=" option 
146 to the 'dig' command. 


For a much more elaborate version of the above script, see Example A-28. 


traceroute 


ping 


Trace the route taken by packets sent to a remote host. This command works within a LAN, WAN, or 
over the Internet. The remote host may be specified by an IP address. The output of this command 
may be filtered by grep or sed in a pipe. 


bash$ traceroute 81.9.6.2 
traceroute to 81.9.6.2 (81.9.6.2), 30 hops max, 38 byte packets 
i ‘ees .xcqlsimaode. com (136. 09.5 1769.8) 191.305 ms 179.4200 ms 179.767 ms 
2 ord). <jlommorio,.cem (336.350.1798) 179.536 ms 179.534 ms 169.685 ms 
S iI192.168,11 LOL (192.109543..103)) — 199.471 me 189.556 me = 


Broadcast an ICMP ECHO REQUEST packet to another machine, either on a local or remote 
network. This is a diagnostic tool for testing network connections, and it should be used with caution. 


bash$ ping localhost 
PING lhocallinost.lecailkcemanin (127,00 .1)) trom 1L275.05051 8 SGH) leüEGS Or cara. 
64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=0 tt1=255 time=709 usec 
64 bytes from localhost.localdomain (127.0.0.1): icmp seq-1 ttl-255 time=286 usec 


=== localhost. lLocalclomeiin plng statistics === 
2 packets transmitted, 2 packets received, 0% packet loss 
round-trip min/avg/max/mdev = 0.286/0.497/0.709/0.212 ms 


A successful ping returns an exit status of 0. This can be tested for in a script. 


HNAME-nastyspammer.com 
4 HNAME=SHOST # Debug: test for localhost. 
count-2 # Send only two pings. 


I 

2 

3 

4 

5 iit [I ping —e Scoume "SimxMESEU J 
6 then 
7 
8 
9 
0 


echo ""SHNAME" still up and broadcasting spam your way." 
else 

echo ""SHNAME" seems to be down. Pity." 
ical 


Perform a DNS (Domain Name System) lookup. The -h option permits specifying which particular 
whois server to query. See Example 4-6 and Example 16-40. 


Retrieve information about users on a network. Optionally, this command can display a user's 
^/.plan,-/.project,and ~/.forward files, if present. 


bash$ finger 


Login Name dne Idle Login Time Office Office Phone 
bozo Bozo Bozeman ttyl 9S Jum 25 1698259 (1:0) 
bozo Bozo Bozeman ttypO Jum 25 Ji 95 (30.0) 


bozo Bozo Bozeman ttypl sein 25) i g77 (30). (00) 


chfn 


vrfy 


bash$ finger bozo 


Login: bozo Name: Bozo Bozeman 
Directory: /home/bozo Shell: /bin/bash 

OWeTices 2555 Clown SE. 5459-1294 

Oa Simoes lien ue Sil 20:13 (MET) om ey 1 hour 38 minutes idle 


On since kri Aue Sil 2Osis (MSI) om jes /O 12 seconds idle 

On since Fri Aug 31 20:13 (MST) on pts/1 

On since Fri Aug 31 20:31 (MST) on pts/2 1 hour 16 minutes idle 
Mail last read Tue Jul 3 10:08 2007 (MST) 

No Plan. 


Out of security considerations, many networks disable finger and its associated daemon. [1] 
Change information disclosed by the finger command. 
Verify an Internet e-mail address. 


This command seems to be missing from newer Linux distros. 


Remote Host Access 


SX, rX 


SZ, YZ 


ftp 


The sx and rx command set serves to transfer files to and from a remote host using the xmodem 
protocol. These are generally part of a communications package, such as minicom. 


The sz and rz command set serves to transfer files to and from a remote host using the zmodem 
protocol. Zmodem has certain advantages over xmodem, such as faster transmission rate and 
resumption of interrupted file transfers. Like sx and rx, these are generally part of a communications 
package. 


Utility and protocol for uploading / downloading files to or from a remote host. An ftp session can be 
automated in a script (see Example 19-6 and Example A-4). 


uucp, uux, cu 


telnet 


uucp: UNIX to UNIX copy. This is a communications package for transferring files between UNIX 
servers. A shell script is an effective way to handle a uucp command sequence. 


Since the advent of the Internet and e-mail, uucp seems to have faded into obscurity, but it still exists 
and remains perfectly workable in situations where an Internet connection is not available or 


appropriate. The advantage of uucp is that it is fault-tolerant, so even if there is a service interruption 
the copy operation will resume where it left off when the connection is restored. 


uux: UNIX to UNIX execute. Execute a command on a remote system. This command is part of the 
uucp package. 


cu: Call Up a remote system and connect as a simple terminal. It is a sort of dumbed-down version of 
telnet. This command is part of the uucp package. 


Utility and protocol for connecting to a remote host. 


® 


The telnet protocol contains security holes and should therefore probably be avoided. 
Its use within a shell script is not recommended. 
wget 
The wget utility noninteractively retrieves or downloads files from a Web or ftp site. It works well in 
a script. 


wget -p http://www.xyz23.com/fileO01.html 
Th jo (oue page-requisite option causes wget to fetch all files 
+ required to display the specified pag 


i 

2 

3 

4 

5 wget -r ftp://ftp.xyz24.net/-bozo/project files/ -O SSAVEFILE 
6 The -r option recursively follows and retrieves all links 
7 
8 
9 
0 
iL 


+ on the specified sit 


wget -c ftp://ftp.xyz25.net/bozofiles/filename.tar.bz2 


[The -c option lets wget resume an interrupted download. 
[This works with ftp servers and many HTTP sites. 


Example 16-42. Getting a stock quote 


! /bin/bash 
quote-fetch.sh: Download a stock quote. 


E NOPARAMS-86 


mie | ex YS’ ] 3 Mw Spsecirty e Stock (Syal) ibo ica 
then echo "Usage: ^basename $0^ stock-symbol" 
exit $E NOPARAMS 

fta 


stock symbol-$1 


file suffix-.html 

Fetches an HTML file, so name it appropriately. 
URL-'http://finance.yahoo.com/q?s-' 

Yahoo finance board, with stock query suffix. 


NPRPRPPRP PPP PY 
O (o 0» -1 O Oi i5 (). NM P. O xo 0 -1 O O1 4S C Io ES 


wget © $(stock symbolj$(file suffix) "S(URL)$(stock symbol)" 


NNN PN 
dex (eS) [Sp (S 


To look up stuff on http://search.yahoo.com: 


N 
[91] 


URL="http://search.yahoo.com/search?fr=ush-newsép=S {query}" 
wget -O "Ssavefilename" "S{URL}" 


N N 
o>) 


N 
[99] 


Saves a list of relevant URLs. 


WN 
y Wo) 


Sil eae SF 


w w 
w N 


Exercises: 


w w 
ow 


w 
Oo 


1) Add a test to ensure the user running the script is on-line. 
(üsBbews § Jexeuerexe icles output (ut Wor) ax ew Vj @ie Ve@inovercic ¢! 


w w CO 
wo e -—1 


2) Modify this script to fetch the local weather report, 
+ taking the user's zip code as an argument. 


[i 
e» 


lynx 


rlogin 


rsh 


rcp 


rsync 


See also Example A-30 and Example A-31. 


The lynx Web and file browser can be used inside a script (with the -dump option) to retrieve a file 
from a Web or ftp site noninteractively. 


1 lynx -dump http://www.xyz23.com/fileOl.html >SSAVEFILE 
With the -traversal option, lynx starts at the HTTP URL specified as an argument, then "crawls" 
through all links located on that particular server. Used together with the -craw1 option, outputs 
page text to a log file. 


Remote login,initates a session on a remote host. This command has security issues, so use ssh 
instead. 


Remote shell,executes command(s) on a remote host. This has security issues, so use ssh 
instead. 


Remote copy, copies files between two different networked machines. 
Remote synchronize, updates (synchronizes) files between two different networked machines. 


bash$ rsync -a ~/sourcedir/*txt /nodel/subdirectory/ 


Example 16-43. Updating FC4 


1 #!/bin/bash 

2 fc4upd.sh 

3 

4 Script author: Frank Wang. 

E Slight stylistic modifications by ABS Guide author. 

6 Used in ABS Guide with permission. 

7 

8 

9 Download Fedora Core 4 update from mirror site using rsync. 
10 Should also work for newer Fedora Cores -- 5, 6, 

il Only download latest package if multiple versions exist, 
12 #+ to save space. 

13 

14 URL-rsync://distro.ibiblio.org/fedora-linux-core/updates/ 
15 URL-rsync://ftp.kddilabs.jp/fedora/core/updates/ 

16 URL-rsync://rsync.planetmirror.com/fedora-linux-core/updates/ 
17 

18 DEST=${1:-/var/www/html/fedora/updates/ } 

19 LOG-/tmp/repo-update-$ (/bin/date +%Y-%m-%d) .txt 
20 PID_FILE=/var/run/${0##*/}.pid 
Al 
22 E_RETURN=85 # Something unexpected happened. 
23 
24 
25 General rsync options 
26 -r: recursive download 
217 t: reserve tim 
219) -v: verbose 
28 
30 iere delet xcluded delete-after partiell” 
Sil 
32 # rsync include pattern 


33 # Leading slash causes absolute path name match. 
34 INCLUDE= ( 


35 "/4/i1386/kde-i18n-Chinese*" 


36 # A $ 

37 # Quoting is necessary to prevent globbing. 
38) ) 

39 

40 

41 rsync exclude pattern 

42 Temporarily comment out unwanted pkgs using "£f" 
43 EXCLUDE-( 

44 HAL 

45 /2 

46 eS 

47 /testing 

48 /4/SRPMS 

49 /A/ppc 

50 /4/x86 64 

S /4/i386/debug 

52 "/4/1386/kde-il8n-*" 

De "/4/1386/0penoffice.org-langpack-*" 
54 1/21/3159). 315/88 wenn 

59 A ASSO / eim ey) 

56 "/4/1386/cman-*" 

37 TAL PASS / align e 

58 A // 3b SI e Gpalloxel= ce 

59 "/4/1386/kernel-smp*" 


60 # "/4/i1386/kernel-xen*" 

61 # "/4/1386/xen-*" 

$2. ) 

63 

64 

G5 sues (y 4 

66 # Let pipe command return possible rsync error, e.g., stalled network. 
67 set -o pipefail 4 Newly introduced in Bash, version 3. 
68 

69 TMP-S(TMPDIR:-/tmp)/$(044*/).9$9 # Store refined download list. 

70 creel UI 

7 rm —-f STMP 2>/dev/null 

72 HEETTE # Clear temporary file on exit. 

TS j 

74 

S, 

16 Cheek pici () 4 

77 # Check if process exists. 

78 ade qp es WSDL Ia o idees 

79 echo "PID file exists. Checking ..." 

80 PID-$(/bin/egrep -o "“[[:digit:]]+" SPID FILE 
81 if /bin/ps --pid SPID &>/dev/null; then 

82 echo "Process $PID found. $(044*/) seems to be running!" 
83 /usr/bin/logger -t $(044*/) \ 

84 "Process $PID found. $(044*/) seems to be running!" 
85 exit $E RETURN 

86 ia 

87 echo Vorocess SEID DOr Se(ojeneYele. Stare mew OLOCSSS s o q^ 

88 1 3l 

99 j 

90 

Sil 

92 # Set overall file update range starting from root or SURL, 

93 #+ according to above patterns. 


94 set_range () { 

95 include- 

96 exclude- 

97 cor © ia WS Cuda (HC ]] co 

98 include-"$include --include \"Sp\"" 
99) done 


100 


[i 
«o 


get. 


for jo im VS mci]; clo 
xclude="Sexclud x«eillasgts WU YY 


done 


trieve and refine rsync update list. 

lige ( 4 

echo SS > Siw ivi, |||) di 
echo "Can't write to pid file $PID_FILE" 
exit $E RETURN 


cho -n "Retrieving and refining update list . . ." 


# Retrieve list 'eval' is needed to run rsync as a single command. 
# $3 and $4 is the date and time of file creation. 
# $5 is the full package name. 
previous- 
pre file- 
pre date-0 
eval /bin/nice /usr/bin/rsync \ 
=e Siiaciluick Sexciluce SEE || \ 
exeueepp 'clie.sk ^m Do 
ek "suc $3, Su. SS || X 
sort -k3 | \ 
( while read line; do 
Get seconds since epoch, to filter out obsolete pkgs. 
cur date-$ (dat gl "Sexo Silane, [| avk "duse Sil, $21") sess) 
echo $cur date 


Get file name. 
Wie esL lese (echo Slims | awk Horiwe SSY) 
echo $cur file 


Get rpm pkg name from file name, if possible. 


ise ||| Sete senikey == “ign |) |p 3elareum 
pkg_name=$ (echo $cur_file | sed -r -e \ 
hey (UPS) a) sh) [E 8l 8 1] 593 a 555 [E12 987 1L 9 J 
else 
pkg name- 
E 


# echo $pkg name 


ie || -—. W'eyolisepescewns v ig Eleven # If not a rpm file, 
echo $cur file >> $TMP #+ then append to download list. 
elif [ "$pkg name" != "$previous" ]; then # A new pkg found. 
echo $pre file >> STMP f Output latest file. 
previous-$pkg name # Save current. 


pre date-$cur date 
pre file-$cur file 


elsi [ “Seune_claice” sere "See cata Je Than 
# If same pkg, but newer, 
pre date-S$cur date #+ then update latest pointer. 
pre file-$cur file 
icd 
done 
echo $pre file >> STMP # TMP contains ALL 


#+ of refined list now. 
# echo "subshell-$BASH, SUBSHELL" 


} # Bracket required here to let final "echo $pre file >> STMP" 
# Remained in the same subshell ( 1 ) with the entire loop. 


RET-$? # Get return code of the pipe command. 


ssh 


167 [ "SRET" -ne 0 ] && { 


168 echo "List retrieving failed with code S$RET" 
16%) exit $E RETURN 

1L 7/0) } 

TEL 

IT echo "done"; echo 

TS y 

174 

175 # Real rsync download part. 

176 gee sile () ( 

iT 

178 echo "Downloading..." 

179 /bin/nice /usr/bin/rsync \ 

180 SOPTS N 

181 filter "merge,+/ $TMP" \ 

182 =—excilude Ye \ 

183 SURL $DEST \ 

184 | Juar forna tee SLOG 

185 

186 RET=$? 

187 

188 # iiter marge, t/ LS Crucial For (e aimeeinieioim 
189 # + modifier means include and / means absolute path. 
190 # Then sorted list in $TMP will contain ascending dir name and 
i 9i #+ prevent the following --exclude '*' from "shortcutting the circuit." 
192 

1193 echo "Done" 

194 

1195 rm -f S$PID FILE 2>/dev/null 

196 

197 return SRET 

199 }} 

199 

200 1; ——————- 

201 # Main 

20/2. anie 

203 check pid 

204 set range 

205 get list 

206 get file 

207 RET-$? 

DOS dy ——————- 

209 

23 xx | "SEEN" -ee Q Ip then 

21 /usr/bin/logger -t $(044*/) "Fedora update mirrored successfully." 
212 else 

213 /usr/bin/logger -t ${0##*/} \ 

214 "Fedora update mirrored with failure code: SRET" 
PAS) Ea 

216 

27 weh SiH i 


See also Example A-32. 


#` Using rcp, rsync, and similar utilities with security implications in a shell script may 
not be advisable. Consider, instead, using ssh, scp, or an expect script. 


Secure shell, logs onto a remote host and executes commands there. This secure replacement for 
telnet, rlogin, rcp, and rsh uses identity authentication and encryption. See its manpage for details. 


Example 16-44. Using ssh 
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ls 


# 
# 
US 
HO 


# 
#+ 


ss 


/bin/bash 
remote.bash: Using ssh. 


This example by Michael Zick. 
Used with permission. 


Presumptions: 
iel=2 igati balog Cejrewmeecl ( "Agel" Jj. 
ssh/sshd presumes stderr ('2') will display to user. 


sshd is running on your machine. 
exe Guay “erernel Clisicieilloiemem, aie joreologlolly als, 
and without any funky ssh-keygen having been done. 


Try ssh to your machine from the command-line: 


$ ssh $HOSTNAME 
Without extra set-up you'll be asked for your password. 
enter password 
when done, $ exit 


Did that work? If so, you're ready for more fun. 


Try ssh to your machine as 'root': 


$ ssh -l root SHOSTNAME 
When asked for password, enter root's, not yours. 


Last login: Tue Aug 10 20:25:49 2004 from localhost.localdomain 


Enter 'exit' when done. 


3 


he above gives you an interactive shell. 


Jl 

but that is beyond the scope of this example. 

The only thing to note is that the following will work in 
'single command' mode. 


A basic, write stdout (local) command. 


Sal 


Now the same basic command on a remote machine. 
Pass a different 'USERNAME' 'HOSTNAME' if desired: 
ER-S$ (USERNAME: -$ (whoami) } 

ST-S(HOSTNAME:-$ (hostname) } 


Now excute the above command-line on the remote host, 
with all transmissions encrypted. 


i] =I SquSmm) Sissi} W Je —1 Ww 
The expected result is a listing of your username's home 
directory on the remote machine. 

To see any difference, run this script from somewhere 
other than your home directory. 


In other words, the Bash command is passed as a quoted line 
to the remote shell, which executes it on the remote machine. 
Jim Elis Case, seme Coes " basa —e Wile —LW Y on your behalf. 


For information on topics such as not having to enter a 
password/passphrase for every command-line, see 

man ssh 

man ssh-keygen 


t is possible for sshd to be set up in a 'single command' mode, 


scp 


67 #+ man sshd config. 
68 
69 exit 0 


® Within a loop, ssh may cause unexpected behavior. According to a Usenet post in the 
comp.unix shell archives, ssh inherits the loop's st din. To remedy this, pass ssh 
either the -n or -f option. 


Thanks, Jason Bechtel, for pointing this out. 


Secure copy, similar in function to rep, copies files between two different networked machines, 
but does so using authentication, and with a security level similar to ssh. 


Local Network 


write 
This is a utility for terminal-to-terminal communication. It allows sending lines from your terminal 
(console or xterm) to that of another user. The mesg command may, of course, be used to disable 
write access to a terminal 
Since write is interactive, it would not normally find use in a script. 

netconfig 
A command-line utility for configuring a network adapter (using DHCP). This command is native to 
Red Hat centric Linux distros. 

Mail 

mail 


Send or read e-mail messages. 


This stripped-down command-line mail client works fine as a command embedded in a script. 


Example 16-45. A script that mails itself 


1 #!/bin/sh 

2 self-mailer.sh: Self-mailing script 

3 

4 adr-$(1:-^whoami') # Default to current user, if not specified. 
5 Typing 'self-mailer.sh wiseguy@superdupergenius.com' 

6 #+ sends this script to that addressee. 

7 Just 'self-mailer.sh' (no argument) sends the script 

8 #+ to the person invoking it, for example, bozo@localhost.localdomain. 
9 

10 For more on the ${parameter:-default} construct, 

11 #+ see the "Parameter Substitution" section 

12 #+ of the "Variables Revisited" chapter. 

13 

14 

15 cat SO || mail -s Weeienjoye WU basencne SO” \ has mailed itseli to you,” "US egheU 
LG 

Ly 

18 

is) Greetings from the self-mailing script. 
20 A mischievous person has run this script, 
21 #+ which has caused it to mail itself to you. 
22 Apparently, some people have nothing better 
23 #+ to do with their time. 


24 f 


25 

2G icing "We date , excsgox WU basename (0 WU manr ledi tons adru 

27 

28 exit 0 

29 

30 # Note that the "mailx" command (in "send" mode) may be substituted 
31 #+ for "mail" ... but with somewhat different options. 


mailto 
Similar to the mail command, mailto sends e-mail messages from the command-line or in a script. 
However, mailto also permits sending MIME (multimedia) messages. 

mailstats 
Show mail statistics. This command may be invoked only by root. 


root# mailstats 
SeAMcisciles irom WHS Jam I 20892509 2006 


M msgsfr bytes_from msgsto bytes to msgsrej msgsdis msgsqur Mailer 
4 1682 24118K 0 OK 0 0 0 esmtp 
9 212 640K 1894 ABIL Sabre 0 0 0 local 
M 1894 24758K 1894 2, BAL SALES 0 0 0 

Cc 414 0 


vacation 
This utility automatically replies to e-mails that the intended recipient is on vacation and temporarily 
unavailable. It runs on a network, in conjunction with sendmail, and is not applicable to a dial-up 
POPmail account. 


Notes 


ul 
A daemon is a background process not attached to a terminal session. Daemons perform designated 
services either at specified times or explicitly triggered by certain events. 


The word "daemon" means ghost in Greek, and there is certainly something mysterious, almost 
supernatural, about the way UNIX daemons wander about behind the scenes, silently carrying out their 
appointed tasks. 
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16.7. Terminal Control Commands 


Command affecting the console or terminal 


tput 
Initialize terminal and/or fetch information about it from terminfo data. Various options permit certain 
terminal operations: tput clear is the equivalent of clear; tput reset is the equivalent of reset. 
bash$ tput longname 
xterm terminal emulator (X Window System) 
Issuing a tput cup X Y moves the cursor to the (X, Y) coordinates in the current terminal. A clear to 
erase the terminal screen would normally precede this. 
Some interesting options to tput are: 
0 bold, for high-intensity text 
9 smul, to underline text in the terminal 
9 smso, to render text in reverse 
0 sgr0, to reset the terminal parameters (to normal), without clearing the screen 
Example scripts using tput: 
1. Example 35-13 
2. Example 35-11 
3. Example A-44 
4. Example A-42 
5. Example 27-2 
Note that stty offers a more powerful command set for controlling a terminal. 
infocmp 
This command prints out extensive information about the current terminal. It references the terminfo 
database. 
bash$ infocmp 
# Reconstructed via infocmp from file: 
/usr/share/terminfo/r/rxvt 
rxvt|rxvt terminal emulator (X Window System), 
am, bce, eo, km, mir, msgr, xenl, xon, 
colors#8, cols#80, it#8, lines#24, pairs#64, 
acsc= "aaffggjjkkllmmnnooppagrrssttuuvvwwxxyyzz((ll))--, 
bel-^G, blink=\E[5m, bold=\E[1m, 
eaae WS 2251. 
clear=\E[H\E[2J, cnorm=\E[?25h, cr-^M, 
reset 
Reset terminal parameters and clear text screen. As with clear, the cursor and prompt reappear in the 
upper lefthand corner of the terminal. 
clear 


The clear command simply clears the text screen at the console or in an xterm. The prompt and cursor 
reappear at the upper lefthand corner of the screen or xterm window. This command may be used 
either at the command line or in a script. See Example 11-25. 

resize 
Echoes commands necessary to set $STERM and STERMCAP to duplicate the size (dimensions) of the 
current terminal. 


bash$ resize 

set noglob; 

setenv COLUMNS '80'; 
setenv LINES '24'; 
unset noglob; 


script 
This utility records (saves to a file) all the user keystrokes at the command-line in a console or an 
xterm window. This, in effect, creates a record of a session. 
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16.8. Math Commands 


"Doing the numbers" 


factor 


bc 


Decompose an integer into prime factors. 


bash$ factor 27417 
Zea UALS ER: 5. 09) ALS) Sh) 


Example 16-46. Generating prime numbers 


NPRPRPPP PPP PY 
O (o 0» -1 O Oi i (). NM P. O xo 0 -1 O O1 i5 C) I ES 
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#!/bin/bash 
# primes2.sh 


# Generating prime numbers the quick-and-easy way, 
#+ without resorting to fancy algorithms. 


CEILING=10000 # 1 to 10000 
ig 


local factors 
ractore=( Seco G1) ) W^ Hoal OUNCE (NE racro MALO Glos 


abr [p 2 WENDEN e 7 

# Third element of "factors" array: 

#+ S{factors[2]} is 2nd factor of argument. 

u^ dE che aS} olanik, eren Costra aS ime) AWe itrexe TE OUS 
#+ and the argument is therefore prime. 


then 

return SPRIME # 0 
else 

return SE_NOTPRIME # null 
EL 
} 
echo 
for n in $(seq SCEILING) 
do 

if is prime $n 

then 

jeretioteie Se Sick 

ipa d ^ Five positions per number suffices. 
done # For a higher S$CEILING, adjust upward, as necessary. 
echo 
exit 


Bash can't handle floating point calculations, and it lacks operators for certain important mathematical 
functions. Fortunately, bc comes to the rescue. 


Not just a versatile, arbitrary precision calculation utility, bc offers many of the facilities of a 
programming language. 


bc has a syntax vaguely resembling C. 

Since it is a fairly well-behaved UNIX utility, and may therefore be used in a pipe, bc comes in handy 
in scripts. 

Here is a simple template for using bc to calculate a script variable. This uses command substitution. 


variable-$ (echo "OPTIONS; OPERATIONS" | bc) 


Example 16-47. Monthly Payment on a Mortgage 


1 #!/bin/bash 

2 monthlypmt.sh: Calculates monthly payment on a mortgage. 

3 

4 

E This is a modification of code in the 

6 #+ "mcalc" (mortgage calculator) package, 

7 #+ by Jeff Schmidt 

8 #+ and 

9 #+ Mendel Cooper (yours truly, the author of the ABS Guide). 
10 http://www.ibiblio.org/pub/Linux/apps/financial/mcalc-1.6.tar.gz [15k] 
JL 

12 echo 

13 echo "Given the principal, interest rate, and term of a mortgage," 
14 echo "calculate the monthly payment." 

15 

16 bottom-1.0 

1L 7 

18 echo 

19 echo -n "Enter principal (no commas) " 

20 read principal 

21 echo -n "Enter interest rat (percent) ^ X at AS, enter Wile", mee Ye . 
22 read interest_r 

23 echo -n "Enter term (months) " 

24 read term 
25 
26 
27  sumcerest r=9 lecto "Sscalle=92 Simcecestc_e/lO0.0” | be) # Cemwex ico decimal 
28 + DOESN E SA e Tox NO 
DI # "scale" determines how many decimal places. 
30 
Zi aimeerasc_reice=S (echo "eeale-9s Sintersst i/12 + 1.0” | be) 
32 
33 
34 top-$(echo "scale-9; $principal*$interest rate^$term" | bc) 
35 # ISSN SCARE A ARETE SEEN ES GTN oa Mna tius 
36 # Standard formula for figuring interest. 
37 
38 echo; echo "Please be patient. This may take a while." 
38 

40 let "months = Sterm - 1" 

41 # 

42 irom ((k=Sinomelase sx > p s=——) )) 

43 do 

44 locis (echo “Sezile=HOp Sameceresic_ sce *Sx'! | lex) 

45 bowtom=s (echo “scalle=9-" Sbottom+scbotr” | be) 

46 4$ bottom = $(($bottom + Sbot")) 

47 done 

48 # 


49 


50 

Sl Rick Boivie pointed out a more efficient implementation 

52 #+ of the above loop, which decreases computation time by 2/3. 
59 

54 for ((x=1; x <= Smonths; x++)) 

55 do 

56 lgoricemi=S scho VUsoalec9; Slsowicow ^ Simcerest race + LV" | lx) 
5 done 

58 

59 

60 And then he came up with an even mor fficient alternative, 
61 #+ one that cuts down the run time by about 95%! 

62 

63 bottom=` { 

64 echo Usemdie-S9r IsonowSoionnp 3Uperest, xeure-Samitesemu, ien 
65 irene (Gzip s <= Smoon mah A 

66 do 

67 echo 'bottom = bottom * interest rate + 1' 

68 done 

69 echo 'bottom' 

70 ip |, doer # Embeds a 'for loop' within command substitution. 
TL 

UZ On the other hand, Frank Wang suggests: 

73 bottom-$(echo "scale=9; (Sinterest rate^$term-1)/(S$interest rate-1)" | bc) 
74 

iS Because 6 

16 The algorithm behind the loop 

77 #+ is actually a sum of geometric proportion series. 

78 Wie Sin formula is S10) (Ea) / (L-6) p 

79 #+ where e0 is the first element and q=e(n+1) /e(n) 

80 #+ and n is the number of elements. 

81 

82 

83 


84 # let "payment = Stop/Sbottom" 
85 payment=$ (echo "scale-2; $top/$bottom" | bc) 
86 # Use two decimal places for dollars and cents. 


88 echo 
89 echo "monthly payment = \SSpayment" # Echo a dollar sign in front of amount. 
90 echo 


il 
92 
93 exa 0 
94 
95 
96 # Exercises: 
OI d 1) Filter input to permit commas in principal amount. 
98 # 2) Filter input to permit interest to b ntered as percent or decimal. 
J9 ie 3) If you are really ambitious, 
100 #+ expand this script to print complete amortization tables. 


Example 16-48. Base Conversion 


1 #!/bin/bash 

2 HERE EEE EE HE EEE EE EEE HE EE EEE EE EEE HE HE EEE EE HE HE EE ER EE EE EE EE EE EE HH EE HEE HE HF 
3 Shellscript: base.sh - print number to different bases (Bourne Shell) 
4 Author : Heiner Steven (heiner.steven@odn.de) 

5 Date : O70 3=95) 

6 Category : Desktop 

7 Sis Dasa gin,y 132 2000/02/06 19255235 halier meg $ 

8 ==> Above line is RCS ID info. 
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echo "SPN = print number to different bases, $VER (stv 


usage: $PN [number 


.] 


If no number is given, the numbers are read from standard i 


A number may be 


binary (base 2) starting with 0b (i.e. 0 
octal (base 8) SHEGueiEzUaS wl © ^ (Gor. Quid 
hexadecimal (base 16) Sireicciine, wrap (ee (Gb ~e, e 
decimal otherwise (i.e. 12)" >&2 
exit SNOARGS 
) # ==> Prints usage message. 
Mec ()) a 
ítgue b # ==> in [list] missing. Why? 
Go exeo "Sena Su" $7 
done 
} 
racal (0 1 Meg "Ss emailt Gor | 
PrintBases () { 
# Determine base of the number 
iHe" GL. s == um. ee wore os c 
do # ==> so operates on command-line arg(s). 
(eere: "SEL shin 
0b*) alleyasiem e F # binary 
Ox* | [a-£]* | [A-F] *) ibase=16;; # hexadecimal 
0*) ibase-8;; # octal 
[1-9] *) ibase=10;; # decimal 
29 
Msg "illegal number $i ignored" 
continue;; 
esac 
# Remove prefix, convert hex digits to uppercase (bc n 
igiolinleysie—" echo WEE || melo ve^ [lemexlgsU doe Viper] V 
# ==> Uses ":" as sed separator, rather than "/". 
# Convert number to decimal 
dec-'echo "ibase-$ibase; $number" | bc^ ==> ‘lope! de c 
case "Sdec" in 
LOO), P number ok 
um continue;; error: ignore 


esac 


# Print all conversions in one line. 
# ==> 'here document' feeds command list to 'bc'. 


exeo "loe <<! 


obase=16; "hex-"; Sdec 
obase=10; "dec="; Sdec 


obase=8; He 


(cgo Selae 


0.27) 


2 


HEE EE HEHE EE EEE EE HH HH HE ERE EE HE EEE EE ERE ERE EEE RE EE HR HE EEE EE EEE HE EE HEE EEH 
Description 
Changes 
21-)3-95 Sw fixed error occuring with Oxb as input ( 
JTEREEREEERERREEREREREEEREEEREREREEEREREREEEREREHRER EE ER EE EEE RE EE ERE ER EE EE HE HE EE HE EEH 
==> Used in ABS Guide with the script author's permission. 
==> Comments added by ABS Guide author. 

NOARGS=85 

PN-'basename "$0" # Program name 

wig celo 'SRewaLSieme 1.2 S" | emu —g V —L2' s; ==> WiR=Il - 

Usage () ( 


Mo) 


IONE, c 


51100) 


lis} elas) c 
[Ay V^ 


allewllaicor witty. 


75, obase-2; "bin="; Sdec 

76 ! 

3g | sed -e 's: : sj” 

78 

7$ done 

80 } 

81 

92. waila | Sir exe (0) | 

83 # ==> Is a "while loop" really necessary here, 

84 # ==>+ since all the cases either break out of the loop 
85 # ==>+ or terminate the script. 

86 # ==> (Above comment by Paulo Marcel Coelho Aragao.) 
87 do 

88 cage VSI aia 

89 ==) Smin oeeie ph 

90 =In)) Usage; ; # ==> Help messag 
91 ZUR Usage;; 

92 *) break;; # First number 
93 esac # ==> Error checking for illegal input might be appropriate. 
94 shift 

95 done 

96 

O7] ii || Sy E © | 

98 then 

99 PrintBases "$Q" 
100 else # Read from stdin. 
101 while read line 
102 do 
103} PrintBases Sline 
104 done 
105 iral 
106 
1,0) 7/ 
108 exit 


An alternate method of invoking bc involves using a here document embedded within a command 
substitution block. This is especially appropriate when a script needs to pass a list of options and 


commands to bc. 


variable-'bc << LIMIT STRING 
options 

statements 

operations 

LIMIT STRING 


o9 noo 


variable-$(bc «« LIMIT STRING 
options 

statements 

operations 

LIMIT STRING 

) 


|Oombpmpmpmowmna 
OY O1 4A (Q0 I9. IP. O x0 O0 -1 O OI 4S (Q) IN S 


Example 16-49. Invoking bc using a here document 


1 #!/bin/bash 
2 # Invoking 'bc' using command substitution 
3 4 in combination with a 'here document'. 


varl-'bc << EOF 
3.33 v 19.798 
EOF 


echo $varl # 362.56 


uw S9 oso ) MOAN alsa worka, 
vl=23.53 
v2=17.881 
v3=83.501 
v4=171.63 


ISSN H H H H H H H 1 
@) Wer (ec. s] fex, (Onp dex. Go NS [23 E Wer (e9 SI (ex; (nb Wes 


var2-$ (bc «« EOF 


scale 4 
Zi. m e ( pyl  9w2 
22 1 = ( Sw * Sywé ) 
AS m 16 4 iS, 35 
24 EOF 
ZI 
26 echo $var2 # 593487.8452 
27 
28 
var3=S (bc -1 << EOF 
scale = 9 
Sie CENE uS 
EOF 


Returns the sine of 1.7 radians. 
Weve; YI gorio calls ths Voci meci ilallonestia\y, 
echo $var3 4 .991664810 


CO CO CO CO CO CO CO CO CO CO DN 
We} (eer <a) py Gl UST (08) INS) [SY Te) Xe} 


Nong tiy GUE igh Zl CUNCTI c n 
40 hypotenuse () # Calculate hypotenuse of a right triangle. 
a q i, (p e Sorc m vA <p OZ. 4) 
42 hyp-$(bc -1 << EOF 
43 scale = 9 
44 gout ( Sil e Si a $92 * $2) 
45 EOF 
46 ) 
47 Can't directly return floating point values from a Bash function. 
48 But, can echo-and-captur 
49 echo "Shyp" 
SO j 
Sl 
52 hyp-$ (hypotenuse 3.68 7.31) 
53 echo "hypotenuse = S$hyp" 4 8.184039344 
54 
55 
DG esae 0) 


Example 16-50. Calculating PI 


#!/bin/bash 
# cannon.sh: Approximating PI by firing cannonballs. 


# Author: Mendel Cooper 
# License: Public Domain 
n^ Wiewesakex 2.2, seeikckeee: ilsiecieOis} c 
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This is a very simple instance of a "Monte Carlo" simulation: 
+ a mathematical model of a real-life event, 
+ using pseudorandom numbers to emulate random chance. 


Consider a perfectly square plot of land, 10000 units on a side. 
This land has a perfectly circular lake in its center, 
+ with a diameter of 10000 units. 
The plot is actually mostly water, except for land in the four corners. 
(Think of it as a square with an inscribed circle.) 


We will fire iron cannonballs from an old-style cannon 
+ at the square. 
All the shots impact somewhere on the square, 
+ either in the lake or on the dry corners. 
Since the lake takes up most of the area, 
+ most of the shots will SPLASH! into the water. 
Just a few shots will THUD! into solid ground 
+ in the four corners of the square. 


If we take enough random, unaimed shots at the square, 
+ Then the ratio of SPLASHES to total shots will approximate 
+ the value of PI/4. 


[The reason for this is that the cannon is actually shooting 
* only at the upper right-hand quadrant of the square, 
+ i.e., Quadrant I of the Cartesian coordinate plane. 

(The previous explanation was a simplification.) 


Theoretically, the more shots taken, the better the fit. 
However, a shell script, as opposed to a compiled language 
+ with floating-point math built in, requires a few compromises. 
This tends to lower the accuracy of the simulation. 
IMENSION-10000 Length of each side of the plot. 
Also sets ceiling for random integers generated. 
AXSHOTS=1000 Fire this many shots. 
10000 or more would be better, but would take too long. 
MULTIPLIER-4.0 Scaling factor to approximate PI. 
clar r M PI-3.141592654 
Actual 9-place value of PI, for comparison purposes. 


et random () 

EED-$ (head -n 1 /dev/urandom | od -N 1 | awk YI print $2 )') 

ANDOM-S$SEED # From "seeding-random.sh" 
#+ example script. 

et "rnum = SRANDOM % SDIMENSION" # Range less than 10000. 

cho $rnum 


istance= # Declare global variable. 

ypotenuse () # Calculate hypotenuse of a right triangle. 
# From "alt-bc.sh" example. 

istance=S (bc -1 << EOF 

cale = 0 

eg ( Bil € Sil d 92 Be ) 


EOR 


Setting "scale" to zero rounds down result to integer value, 
+ a necessary compromise in this script. 
This decreases the accuracy of the simulation. 


dal 


T 

# main() { 

# "Main" code block, mimmicking a C-language main() function. 
# Initialize variables. 


shots=0 
splashes=0 
thuds=0 
Pi=0 
error=0 


while [ 
do 


Seine Ihc WIM IOI ] 


xCoord=$ (get random) 
yCoord-$ (get random) 
hypotenuse $xCoord $yCoord 


(enora) 


"#54, W Sent s 

Uke = ae U Stool 

"Yc = $4d ^" SyCoord 
UpPpisicancs = Bac WY Schisicenves 


{OIE SLIME dE 
printf 
joe iia se 
printf 


iif [ 

then 
echo -n "SPLASH! y 
( (splashes++)) 

else 
echo -n "THUD! " 
((thuds++) ) 

i3 


-le "SDIMENSION" ] 


NSGhist eames” 


Pi=S (echo "scale=9; 
# Multiply ratio by 4.0. 
Clio: io) WIP ASPRA 

echo 


done 
echo 


echo "After Sshots shots, 
# Tends to run a bit high, 


SPMULTIPLIER*Ssplashes/S$shots" | 


PI looks like approximately 


# Main loop. 


# Get random X and Y coords. 


# Hypotenuse of 


#+ right-triangle = distance. 


# Distance from 
center of lake 

== DEINE OSIK EE o == 
coordinate (0,0). 


bc) 


Spi" 


#+ probably due to round-off error and imperfect randomness of S$RANDOM. 


# But still usually within plus-or-minus 5$ 
#+ a pretty good rough approximation. 
error-$(echo "scale-9; SPi - $M PI" | 
pct, error-$ (echo "scale-2; 


bc) 


echo -n "Deviation from mathematical value of PI - 
acing " (Se orror: eu) 

echo 

# End of "main" code block. 

# } 

# 


100.0 “= Sartor / EMDI | 


be) 
Serror" 


exit 


# One might well wonder whether a shell script is appropriate for 
#+ an application as complex and computation-intensive as a simulation. 


dc 


139 # 

140 # There are at least two justifications. 

141 # 1) As a proof of concept: to show it can be done. 
# 
# 


142 2) To prototype and test the algorithms before rewriting 
143 #+ it in a compiled high-level language. 


See also Example A-37. 


The de (desk calculator) utility is stack-oriented and uses RPN ("Reverse Polish Notation"). Like be, 
it has much of the power of a programming language. 


Techo Vy B5 = jo | cle # 56 
2 # Pushes 7, then 8 onto the stack, 
3 #+ multiplies ("*" operator), then prints the result ("p" operator). 


Most persons avoid dc, because of its non-intuitive input and rather cryptic operators. Yet, it has its 
uses. 


Example 16-51. Converting a decimal number to hexadecimal 


!/bin/bash 
hexconvert.sh: Convert a decimal number to hexadecimal. 


E_NOARGS=85 Command-line arg missing. 


al 

2 

3 

4 

5 BASE-16 Hexadecimal. 

6 

7 su [| =2 VSL” | 

8 then Need a command-line argument. 

9 echo "Usage: $0 number" 

10 exit $E NOARGS 

JL3L pu Exercise: add argument validity checking. 
1,2 

13 

14 hexcvt () 

15 4 

U6 ax [p —- US" ] 

17 then 

18 echo 0 

Le return # "Return" 0 if no arg passed to function. 
2(0 Ea 
AL 
22 woimo "WISI Vemm gy s" || cle 
23 # [9] sets radix (numerical base) of output. 
24 # [9 quus Tie TOS Gu SEG. 
25 # For other options: 'man dc' 
26 return 
29 y 
28 
29) Ines MSIL 
30 
Sil exit 


Studying the info page for dc is a painful path to understanding its intricacies. There seems to be a 
small, select group of dc wizards who delight in showing off their mastery of this powerful, but 
arcane utility. 


bash$ echo "16i[q]sa[1n0=aln100%P1n100/sn1bx] sbA0D68736142sn1lbxq" | dc 
Bash 


awk 


il de <<< 3 swisc2/] i 1.618905350667 

2 Sumo Feed operations to dc using a Here String. 

3 $5 Pushes 10 and sets that as the precision (10k). 

4 Quo Pushes 5 and takes its square root (5v, v = square root). 
5 rs Bushes db ancl acle ait to ths semone leo. (b) s 

6 ^^ Pushes 2 and divides the running total by that (2/). 

ji ^ Pops and prints the result (p) 

8 The result is 1.6180339887 

E which happens to be the Pythagorean Golden Ratio, to 10 places. 


Example 16-52. Factoring 


!/bin/bash 
factr.sh: Factor a number 


IN=2 # Will not work for number smaller than this. 
E_NOARGS=85 
E TOOSMALL-86 


ac [ =a Si ] 

then 
echo "Usage: $0 number" 
exit $E NOARGS 

ie aL 


ai [ MSY aie Veni ] 

then 
echo "Number to factor must be SMIN or greater." 
exit $E TOOSMALL 

fal 


NPRPRPPRP PPP PY 
O io 0» -1 O Oi i$ (0. M. IS. O xo 0 -1 O O1 i CQ) I E 


Exercise: Add type checking (to reject non-integer arg). 


Bil 

22 ChG “mecrors oie Gig" 

23 

24 echo "$1[p]s2[lip/dli£$0-1dvsr]s12sid2£0-13sidvsr[dli$0-N 

25 JJli2s98372.]89.:«2" || cle 

26 

27 Above code written by Michel Charpentier <charpov@cs.unh.edu> 
28 (as a one-liner, here broken into two lines for display purposes). 
29 Used in ABS Guide with permission (thanks!). 

30) 

31 exit 

92 

Sg y S Gla tac. gia 27/1 59 

34 # 2 

SS 4 a 

36 # 11 

St 4; 4093 


Yet another way of doing floating point math in a script is using awk's built-in math functions in a 
shell wrapper. 


Example 16-53. Calculating the hypotenuse of a triangle 


1 #!/bin/bash 

2 4 hypotenuse.sh: Returns the "hypotenuse" of a right triangle. 
SEE (square root of sum of squares of the "legs") 
4 


5 ARGS-2 # Script needs sides of triangle passed. 
6 E BADARGS-85 # Wrong number of arguments. 
7 
8 if [ $4 -ne "SARGS" ] 4 Test number of arguments to script. 
9 then 

LO echo "Usage: 'basename $0° side 1 side 2" 

iil exit SE_BADARGS 

U2) sea 

13 

14 

15 AWKSCRIET=" | primei YSS.7E\WmY, Sore Sll + $2992) ) P v 
Le command (s) / parameters passed to awk 

37 

18 

ig Now, pipe the parameters to awk. 

20 echo -n "Hypotenuse of $1 and $2 = " 

2 echo $1 $2 | awk "SAWKSCRIPT" 

22 SLO E ES PRG NE 


29 An echo-and-pipe is an easy way of passing shell parameters to awk. 


24 

25 exit 

26 

27 4 Exercise: Rewrite this script using 'bc' rather than awk. 

Z8 i Which method is more intuitive? 
Prev Home Next 
Terminal Control Commands Up Miscellaneous Commands 


Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Chapter 16. External Filters, Programs and Commands Next 


16.9. Miscellaneous Commands 


Command that fit in no special category 


jot, seq 
These utilities emit a sequence of integers, with a user-selectable increment. 


The default separator character between each integer is a newline, but this can be changed with the -s 
option. 


bash$ seq 5 
il 


Ow WN 


bash$ seq -s : 5 
qeu 


Both jot and seq come in handy in a for loop. 


Example 16-54. Using seq to generate loop arguments 


#!/bin/bash 
# Using "seq" 


echo 
for em im "see GU i Gu for 2 im S( ees sa ) 
# Same as one mL stig db 2. Gr AY O (saves much typing!). 
# May also use 'jot' (if present on system). 
do 
Seno =m MEL Y 
done F23 A5 5 80 


# Example of using the output of a command to generate 
s ime Plier am a Vitor’ Logg, 


echo; echo 


COUNT-80 # Yes, 'seq' also accepts a replaceable parameter. 


N oL pop pp pppÀpGÀG: 
O (o 0» -1 O Oi i (). M IS. O «(o 0 -1 O O1 i Q I E 


for a in “seq $COUNT' # or for a in $( seq SCOUNT ) 


2L ClO) 

22 exo =m "eg Y 

23 done if 2 Ss S eno 80 

24 

25 echo; echo 

26 

27 BEGIN=75 

28 END=80 

28, 

30 for a in “seq SBEGIN SEND 

Sil Giving "seq" two arguments starts the count at the first one, 
32 #+ and continues until it reaches the second. 
33) Clo) 


34 acne =i Weg U 


35 
36 
oy 
38 
3 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Syl 
52 
53 


done uc 45» US qu WS WS SO 


BEGIN=45 
INTERVAL=5 
END=80 
for a in “seq SBEGIN SINTERVAL S$END' 
Giving "seq" thr arguments starts the count at the first one, 
+ uses the second for a step interval, 
+ and continues until it reaches the third. 
do 
Sela. ia Ser W 
done i? 45 50 55. 60 65 70 75 9 
echo; echo 
exit 0 


A simpler example: 


[= (€» We) (G9 c (Cx (Oa de c» 9 p 


PR 


# 
#+ 
GO 


PR 


fo 
do 


do 


Create a set of 10 files, 

joyeux iade. ubl 2- 2 5 v tado dic 
UNT-10 

EFIX-file 


r filename in 'seq $COUNT' 


touch S$PREFIX.$filename 

# Or, can do other operations, 
#+ such as rm, grep, etc. 

ne 


Example 16-55. Letter Count" 


NPRPRPRPRP PPP PY 
O io 00 -1 O Qi i (). NM P. O xo 0 - O O1 i CQ Io E 


NNN 
w N ES 


24 
25 
26 
221) 


! 


Ea 


le 


sh 


# 
Bit 


INARGS-2 


/bin/bash 


letter-count.sh: Counting letter occurrences in a text file. 


Written by Stefano Palmeri. 
Used in ABS Guide with permission. 
Slightly modified by document author. 


BADARGS=65 
LE=$1 


# Script requires at least two arguments. 


t LETTERS=S$#-1 # How many letters specified (as command-line args). 


# (Subtract 1 from number of command-line args.) 


ow_help() { 
echo 


echo Usage: ~basename $0' file letters 
echo Note: ^basename $0' arguments are case sensitiv 


echo Example: ^basename $0` 
echo 


Checks number of arguments. 

[ $4 -lt SMINARGS ]; then 
echo 

echo "Not enough arguments." 
echo 


OGA Sse (Gi am UP JE, WW W se, 


getopt 


28 
29 
30 
31 
32 
33 
34 
25 
36 
SY 
38 
3 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sil 
52 
52 
54 
55 


show_help 


exit $I 


ital 


# Checks if file exist 


xi [ u ert SIP IIL 
echo "File \ 
exit $E BA 

iE aL 


# Counts letter occurrences 
for m Fin SEG SIL; 


shift 
xd [| 
else 
ia. 
done 
exit $? 


E BADARGS 


]; the 
iU STO TEE 


DARGS 


ETTERS 
"melee =m U 
echo "$1" 
ech 


So 
n 


X" does imo exist.” 


"g tele 


Eu 
NS 


we —e -eg X gp then # Checks arg. 
"sug GUTES | te -ecl "SI" | we -e° s Counting. 


© VS 26 io c Single chu." 


# This script has exactly the same functionality as letter-count2.sh, 
xecutes faster. 


#+ but 
# Why? 


°) Somewhat more capable than seq, jot is a classic UNIX utility that is not normally 
included in a standard Linux distro. However, the source rpm is available for 
download from the MIT repository. 


Unlike seq, jot can generate a sequence of random numbers, using the -r option. 


1069 
12:02 
1428 


bash$ jot -r 3 999 


The getopt command parses command-line options preceded by a dash. This external command 
corresponds to the getopts Bash builtin. Using getopt permits handling long options by means of the 
-1 flag, and this also allows parameter reshuffling. 


Example 16-56. Using getopt to parse command-line options 


| o ombpmo 
( NM HP O0 -10 0 4&0 NPd| 


S 


wa nuu umum U 


B Bm gr GE gm qr gw Qm 


!/bin/bash 
Using getopt 


sh 
sh 
sh 
sh 
sh 
sh 
sh 
sh 
sh 


=a 
abe 
=a co 
=Q] 
ODN 
c(l NOUS 
-abcd 
-abcdZ 
-zZ 


Se 


Try the following when invoking this script: 
ex33a. 
ex33a. 
ex33a. 
ex33a. 
ex33a. 
ex33a. 
ex33a. 
ex33a. 
ex33a. 


14 sh ex33a.sh a 

15 Explain the results of each of the above. 

16 

17 E OPTERR-65 

18 

19 aie | "S5" eeg 9 | 

20 then # Script needs at least one command-line argument. 


2 sco "sese 9) =-loptiones mc] 


22 exit $E OPTERR 

23 3E3L 

24 

25 Egit == "grew YaloeclsY WE)" 


26 4 Sets positional parameters to command-line arguments. 
27 4 What happens if you use "S$*" instead of "S@"? 

28 

Be) Weide [p d^ c SW T 

30 do 

Sil case "$1" in 

32 -a) echo “Option wav 

33 -i9) echo "(Orso \Wa\""p E 

34 =E) echo Option Weir 

35 =E) echo Opium VENS SVP 

36 *) break;; 

3 esac 

38 

39 shift 

40 done 

41 

42 # It is usually better to use the 'getopts' builtin in a script. 
43 # See "ex33.sh." 

44 

45 exit 0 


8) As Peggy Russell points out: 
It is often necessary to include an eval to correctly process whitespace and quotes. 
1 args-$(getopt -o a:bc:d -- "$Q") 


2 eval set -— "Sargs" 


See Example 10-5 for a simplified emulation of getopt. 


run-parts 


yes 


The run-parts command [1] executes all the scripts in a target directory, sequentially in ASCII-sorted 
filename order. Of course, the scripts need to have execute permission. 


The cron daemon invokes run-parts to run the scripts in the /etc/cron. * directories. 

In its default behavior the yes command feeds a continuous string of the character y followed by a 
line feed to stdout. A control-C terminates the run. A different output string may be specified, as 
in yes different string, which would continually output different string to 
stdout. 

One might well ask the purpose of this. From the command-line or in a script, the output of yes can be 
redirected or piped into a program expecting user input. In effect, this becomes a sort of poor man's 
version of expect. 


yes | fsck /dev/hdal runs fsck non-interactively (careful!). 


yes | rm -r dirname has same effect as rm -rf dirname (careful!). 


Caution advised when piping yes to a potentially dangerous system command, such as 
fsck or fdisk. It might have unintended consequences. 


8^; The yes command parses variables, or more accurately, it echoes parsed variables. For 
example: 


bash$ yes SBASH VERSION 
3.1.17(1)-release 
1)-release 
1)-release 
1)-release 
1)-release 


w CO CO CO 


Jo 357 4 
o dbz dU ( 
JL 5307 ( 
oily 4 


This particular "feature" may be used to create a very large ASCII file on the fly: 


bash$ yes $PATH » huge file.txt 
Ctl-C 


Hit Ct1-C very quickly, or you just might get more than you bargained for. . . . 
The yes command may be emulated in a very simple script function. 


A ses () 

2 ( # Trivial emulation of "yes" 

3 local DEFAULT TEXT-"y" 

4 walle | crre | # Endless loop. 
5 do 
6 
J 
8 


aie [| ew S | 
then 
Aelng "HSIDISEPRNUIEP. Iai 

9 else # If argument 
10 echo "$1" # ... expand and echo it. 
LL dE ab 
12 done # The only things missing are the 
13 } #+ --help and --version options. 


banner 
Prints arguments as a large vertical banner to stdout, using an ASCII character (default '#'). This 
may be redirected to a printer for hardcopy. 


Note that banner has been dropped from many Linux distros. 
printenv 
Show all the environmental variables set for a particular user. 


bash$ printenv | grep HOME 
HOME-/home/bozo 


B The Ip and Ipr commands send file(s) to the print queue, to be printed as hard copy. [2] These 
commands trace the origin of their names to the line printers of another era. 
bash$ lp filel.txtorbashlp «filel.txt 
It is often useful to pipe the formatted output from pr to Ip. 
bashS pr -options filel.txt | lp 


Formatting packages, such as groff and Ghostscript may send their output directly to Ip. 


bash$ groff -Tascii file.tr | lp 


bash$ gs -options | lp file.ps 


Related commands are Ipq, for viewing the print queue, and Iprm, for removing jobs from the print 


queue. 
tee 
[UNIX borrows an idea from the plumbing trade.] 
This is a redirection operator, but with a difference. Like the plumber's fee, it permits "siphoning off" 
to a file the output of a command or commands within a pipe, but without affecting the result. This is 
useful for printing an ongoing process to a file or paper, perhaps to keep track of it for debugging 
purposes. 
(redirection) 
|——--- dto file 
| 
| 
Conmmanc| => Commence! ———» [tee => comente ==> ==> quiu oit Ploe 
1 cat listfile* | sort | tee check.file | unig > result Tille 
2 # RAKRAKRAKRAKRKRAAKRRAN ^^^ 
S 
4 # The file "check.file" contains the concatenated sorted "listfiles," 
5 #+ before the duplicate lines are removed by ‘'uniq.' 
mkfifo 
This obscure command creates a named pipe, a temporary first-in-first-out buffer for transferring data 
between processes. [3] Typically, one process writes to the FIFO, and the other reads from it. See 
Example A-14. 
1 #!/bin/bash 
2 # This short script by Omair Eshkenazi. 
3 # Used in ABS Guide with permission (thanks!). 
4 
5 mkfifo pipel # Yes, pipes can be given names. 
6 mkfifo pipe2 # Hence the designation "named pipe." 
7 
e (eue GH V a [| ee ama ASA Sioa? Sa & 
ols =a i “oe S TUS e t £39 s pipel | 
IL) iewie cl! " —12. | Tests — je 
JUL 
WZ sen -E Tenljxeul 
lo rmi =E quje 
14 
15 # No need to kill background processes when script terminates (why not?). 
16 
17 axit S7 
1L) 
19 Now, invoke the script and explain the output: 
20 sh mkfifo-example.sh 
21 
22, HES) Tee gz BOZO 
23 pipel BOZO 
24 pipe2 BOZO 
25 mkfifo-example.sh BOZO 
26 Mixed.msg BOZO 
pathchk 


This command checks the validity of a filename. If the filename exceeds the maximum allowable 
length (255 characters) or one or more of the directories in its path is not searchable, then an error 
message results. 


dd 


Unfortunately, pathchk does not return a recognizable error code, and it is therefore pretty much 
useless in a script. Consider instead the file test operators. 


This is the somewhat obscure and much feared data duplicator command. Originally a utility for 
exchanging data on magnetic tapes between UNIX minicomputers and IBM mainframes, this 
command still has its uses. The dd command simply copies a file (or stdin/stdout), but with 
conversions. Possible conversions are ASCII/EBCDIC, [4] upper/lower case, swapping of byte pairs 
between input and output, and skipping and/or truncating the head or tail of the input file. 


1 # Converting a file to all uppercase: 


2 
3 dd if-$filename conv-ucase > $filename.uppercas 
4 d lcase # For lower case conversion 


Some basic options to dd are: 
0 if=INFILE 


INFILE is the source file. 
o ofZOUTFILE 


OUTFILE is the target file, the file that will have the data written to it. 
0 bs-BLOCKSIZE 


This is the size of each block of data being read and written, usually a power of 2. 
9 skip=BLOCKS 


How many blocks of data to skip in INFILE before starting to copy. This is useful when the 
INFILE has "garbage" or garbled data in its header or when it is desirable to copy only a 
portion of the INFILE. 

0 seek=BLOCKS 


How many blocks of data to skip in OUTFILE before starting to copy, leaving blank data at 
beginning of OUTFILE. 
© countZBLOCKS 


Copy only this many blocks of data, rather than the entire INFILE. 
0 convZCONVERSION 


Type of conversion to be applied to INFILE data before copying operation. 
A dd --help lists all the options this powerful utility takes. 


Example 16-57. A script that copies itself 


1 #!/bin/bash 
2 4 self-copy.sh 

3 

4 # This script copies itself. 

5 

6 file_subscript=copy 

7 

8 dd if-$0 of=$0.Sfile_subscript 2>/dev/null 
9 # Suppress messages from dd: ua DIE 


13 # A program whose only output is its own source code 
14 #+ is called a "quine" per Willard Quine. 
15 # Does this script qualify as a quine? 


Example 16-58. Exercising dd 


#!/bin/bash 
# exercising-dd.sh 


# Script by Stephane Chazelas. 
# Somewhat modified by ABS Guide author. 


infile-$0 w^ MOSES): (eXe3edleXE c 
outfile-log.txt # Output file left behind. 
n=3 

p=5 


dd if=Sinfile of=Soutfile bs-1 skip-$((n-1)) count=$((p-n+1)) 2» /dev/null 
i? MEINES Characters m ojo (3 to 5) teem clas Seit. 


echo =n “Slike vorid | lel cissi (rexswvessusdsjloxee 2> Jokes imul 
# Echoes "hello world" vertically. 
# Why? A newline follows each character dd emits. 


i (£3. wer (o9 c (ex, (Cw des ED us. [3 Ee KS Co x] (Cx, Gil dE eS de» (3 


[SOY [SO di 


exi (0) 


To demonstrate just how versatile dd is, let's use it to capture keystrokes. 


Example 16-59. Capturing Keystrokes 


1 #!/bin/bash 

2 # dd-keypress.sh: Capture keystrokes without needing to press ENTER. 
3 

4 

5 keypresses=4 Number of keypresses to capture. 
6 

y 

8 old tty setting-$(stty -g) Save old terminal settings. 

9 
10 echo "Press Skeypresses keys." 
ii atty eanan =Selie Disable canonical mode. 

12 Disable local echo. 

13 keys=$(dd bs=1 count=Skeypresses 2> /dev/null) 

iA aie ViololY Wises) Seok abit Vari (suiayoybhe draLll)) Yen. feyexexealitst yel e 

15) 

IG iw "Selle tey Setia" # Restore old terminal settings. 
al 

18 echo "You pressed the \"Skeys\" keys." 

Ig 
20 # Thanks, Stephane Chazelas, for showing the way. 
21 exit 0 


The dd command can do random access on a data stream. 


echo -n . | dd bs=1 seek-4 of-file conv=notrunc 
# The "conv-notrunc" option means that the output file 
#+ will not be truncated. 


(Gg; dE (Gs) ISS) [S 


i? idees. SoC, 


The dd command can copy raw data and disk images to and from devices, such as floppies and tape 
drives (Example A-5). A common use is creating boot floppies. 


dd if-kernel-image of=/dev/£d0H1440 


Similarly, dd can copy the entire contents of a floppy, even one formatted with a "foreign" OS, to the 
hard drive as an image file. 


dd if=/dev/fd0 ofz/home/bozo/projects/floppy.img 
Other applications of dd include initializing temporary swap files (Example 30-2) and ramdisks 
(Example 30-3). It can even do a low-level copy of an entire hard drive partition, although this is not 


necessarily recommended. 


People (with presumably nothing better to do with their time) are constantly thinking of interesting 
applications of dd. 


Example 16-60. Securely deleting a file 


1 #!/bin/bash 
2 ISILGIE-OMIE SINS lage Vell traces OF a file. 
3 
4 This script overwrites a target file alternately 
5 #+ with random bytes, then zeros before finally deleting it. 
6 After that, even examining the raw disk sectors by conventional methods 
7 #+ will not reveal the original file data. 
8 
9 PASSES=7 Number of file-shredding passes. 
10 Increasing this slows script execution, 
ial + especially on large target files. 
12 BLOCKSIZE=1 I/O with /dev/urandom requires unit block size, 
is) + otherwise you get weird results. 
14 E_BADARGS=70 Various error exit codes. 
15 E_NOT_FOUND=71 
16 E_CHANGED_MIND=72 
jy 
1g sau i —- YS” ] # No filename specified. 
19 then 
20 echo "Usage: 'basename $0! filename" 
2 exit $E BADARGS 
22 3E 
23 
24 file-$1 
25 
AS ade || 8 =e VeSxxIQV 1 
27 then 
28 exeo Vieille WUSrileWw! mere formel Y 
29 exit $E NOT FOUND 
SI) Fi 
Sy 
32 echo; echo -n "Are you absolutely sure you want to blot out \"$file\" (y/n)? " 


Ww 
w 


read answer 


34 case "Sanswer" in 
35 [nN]) echo "Changed your mind, huh?" 


36 exit $E CHANGED. MIND 

37 BE 

38 *) echo UiBlottimg ovt file \"Stile\". "26 

39 esac 

40 

41 

AD selkeiicicio=S (dus. =i Weeds || aw: Ujoresuotc S'S) p mielii 5 ie ila dheyoxenelmg 
43 pass_count=1 

44 

45 chmod utw "$file" # Allow overwriting/deleting the file. 

46 

47 echo 

48 

49 while [ "$pass count" -le "SPASSES" ] 

50 do 

51 echo "Pass 4$pass count" 

52 sync # Flush buffers. 

53 dd if-/dev/urandom of-$file bs-SBLOCKSIZE count=$flength 

54 4 Fill with random bytes. 

55 sync # Flush buffers again. 

56 dd if=/dev/zero of-$file bs=SBLOCKSIZE count-$flength 

57 # Fill with zeros. 

58 sync # Flush buffers yet again. 

5 ret elpasismeounta: sean 

60 echo 

61 done 

62 

63 

64 rm -f Sfile # Finally, delete scrambled and shredded file. 

65 sync # Flush buffers a final time. 

66 

67 echo “Walle \W Sita le\" loue owe aime! clolleceacl,"p echo 

68 

(99 

70 exit 0 

Ta 

72 This is a fairly secure, if inefficient and slow method 

73 #+ of thoroughly "shredding" a file. 

74 The "shred" command, part of the GNU "fileutils" package, 

75 #+ does the same thing, although more efficiently. 

76 

WI The file cannot not be "undeleted" or retrieved by normal methods. 
78 However a 

79 #+ this simple method would *not* likely withstand 

80 #+ sophisticated forensic analysis. 

81 

82 This script may not play well with a journaled file system. 

83 ÓuxewetesLe Ke ((elaviescayeyodlic)) B. e ahi, SO ais closes. 

84 

85 

86 

87 Tom Vier's "wipe" file-deletion package does a much more thorough job 
88 #+ of file shredding than this simple script. 

89 http://www.ibiblio.org/pub/Linux/utils/file/wipe-2.0.0.tar.bz2 
90 

eut For an in-depth analysis on the topic of file deletion and security, 
92 #+ see Peter Gutmann's paper, 

93 #+ "Secure Deletion of Data From Magnetic and Solid-State Memory". 
94 http://www.cs.auckland.ac.nz/-pgut001/pubs/secure del.html 


See also the dd thread entry in the bibliography. 


The od, or octal dump filter converts input (or files) to octal (base-8) or other bases. This is useful for 
viewing or processing binary data files or otherwise unreadable system device files, such as 
/dev/urandom, and as a filter for binary data. 


1 head c /dev/urandom | od -N4 -tu4 | sed -me '1s/.* //p' 
2 4 Sample output: 1324725719, 3918166450, 2989231420, etc. 
3 


4 # From rnd.sh example script, by Stéphane Chazelas 
See also Example 9-16 and Example A-36. 
hexdump 
Performs a hexadecimal, octal, decimal, or ASCII dump of a binary file. This command is the rough 
equivalent of od, above, but not nearly as useful. May be used to view the contents of a binary file, in 
combination with dd and less. 


1 dd if-/bin/ls | hexdump -C | less 
2 4 The -C option nicely formats the output in tabular form. 
objdump 
Displays information about an object file or binary executable in either hexadecimal form or as a 
disassembled listing (with the -d option). 


bash$ objdump -d /bin/ls 
fora ss file format elf32-1386 


Disassembly of section .init: 


OCOLI «S sabio E 
80490bc: 55) push Sebp 
80490bd: 89 e5 mov Sesp, sebp 


mcookie 
This command generates a "magic cookie," a 128-bit (32-character) pseudorandom hexadecimal 
number, normally used as an authorization "signature" by the X server. This also available for use in a 
script as a "quick 'n dirty" random number. 


1 random000=$ (mcookie) 


Of course, a script could use md5sum for the same purpose. 


1 # Generate md5 checksum on the script itself. 
2 random001-'md5sum $0 | awk '(print $1)'^ 
3 # Uses 'awk' to strip off the filename. 


The mcookie command gives yet another way to generate a "unique" filename. 


Example 16-61. Filename generator 


1 #!/bin/bash 

2 # tempfile-name.sh: temp filename generator 

3 

4 BASE STR- mcookie" 32-character magic cookie. 

5 POS-11 Arbitrary position in magic cookie string. 
6 LEN-5 Get SLEN consecutive characters. 

7 

8 prefix=temp This is, after all, a "temp" file. 

9 For more "uniqueness," generate th 

10 + filename prefix using the same method 
1 + as the suffix, below. 

L2 

13 


suffix-$(BASE STR:POS:LEN) 


units 


m4 


14 # Extract a 5-character string, 


LS iu SEAMEN GE Position Ji. 
16 

17 temp filename-$prefix.$suffix 

18 # Construct the filename. 
1$ 

20 echo "Temp filename = "$temp filename"" 

2 


22 4 sh tempfile-name.sh 

23 4 Temp filename = temp.el9ea 

24 

25 # Compare this method of generating "unique" filenames 
26 #+ with the 'date' method in ex51.sh. 

Bl 

28 exit 0 


This utility converts between different units of measure. While normally invoked in interactive mode, 
units may find use in a script. 


Example 16-62. Converting meters to miles 


#!/bin/bash 
# unit-conversion.sh 


Comyeicic Wines (0) # Takes as arguments the units to convert. 


{ 


guess USJU Vee" | eeel silent Vij! || ems UdWuexunewe ERU 
# Strip off everything except the actual conversion factor. 
echo SGT 


Unitl=miles 

Unit2=meters 

ekactor= Convert omita Simic SUmit2 
quantity=3.73 


result=$ (echo $quantity*$cfactor | bc) 


ehg Vihere are Şresult SUmit2 im şeweantcity SUmitil, Y 


NOB pop p PPP PPE 
O (o 0» -1 O Oi i o M IS. O «(o 0 -1 O O1 i CQ I E 


21 # What happens if you pass incompatible units, 
22 #+ such as "acres" and "miles" to the function? 
23 

24 exit 0 


A hidden treasure, m4 is a powerful macro [5] processing filter, virtually a complete language. 
Although originally written as a pre-processor for RatFor, m4 turned out to be useful as a stand-alone 
utility. In fact, m4 combines some of the functionality of eval, tr, and awk, in addition to its extensive 
macro expansion facilities. 


The April, 2002 issue of Linux Journal has a very nice article on m4 and its uses. 


Example 16-63. Using m4 


1 #!/bin/bash 


2 # m4.sh: Using the m4 macro processor 
3 
4 # Strings 
5 string-abcdA01 
(5. &xeleo) Wilein(Sstevesliag))Y [| adl # 7 
7 echo “substr(Sstring,4)" | m4 # A01 
a aho Vresiero(Ssicraling, [O=1]) [LEE] NEZA) || má # O12 
9 
10 # Arithmetic 
dil seha Vaineie(22) || sd # 23 
12 echo Yeyal(99 / 3)” sd # 33 
13 
Jd est 


xmessage 


zenity 


doexec 


dialog 


SOX 


This X-based variant of echo pops up a message/query window on the desktop. 


1 xmessage Left click to continue -button okay 


The zenity utility is adept at displaying GTK+ dialog widgets and very suitable for scripting purposes. 


The doexec command enables passing an arbitrary list of arguments to a binary executable. In 
particular, passing argv [0] (which corresponds to $0 in a script) lets the executable be invoked by 
various names, and it can then carry out different sets of actions, according to the name by which it 
was called. What this amounts to is roundabout way of passing options to an executable. 


For example, the /usr/local/bin directory might contain a binary called "aaa". Invoking doexec 
/usr/local/bin/aaa list would list all those files in the current working directory beginning with an "a", 
while invoking (the same executable with) doexec /usr/local/bin/aaa delete would delete those files. 


£g") The various behaviors of the executable must be defined within the code of the 
executable itself, analogous to something like the following in a shell script: 


case "basename $0^ in 

"namel" ) do something;; 

"name2" ) do something else;; 
"name3" ) do yet another thing;; 
9 ) logi owes A 

esac 


Ox Ole WN ES 


The dialog family of tools provide a method of calling interactive "dialog" boxes from a script. The 
more elaborate variations of dialog -- gdialog, Xdialog, and kdialog -- actually invoke X-Windows 
widgets. 


The sox, or "sound exchange" command plays and performs transformations on sound files. In fact, 
the /usr/bin/play executable (now deprecated) is nothing but a shell wrapper for sox. 


For example, sox soundfile.wav soundfile.au changes a WAV sound file into a (Sun audio format) 
AU sound file. 


Shell scripts are ideally suited for batch-processing sox operations on sound files. For examples, see 
the Linux Radio Timeshift HOWTO and the MP3do Project. 


Notes 


[1] This is actually a script adapted from the Debian Linux distribution. 


The print queue is the group of jobs "waiting in line" to be printed. 

For an excellent overview of this topic, see Andy Vaught's article, Introduction to Named Pipes, in the 
September, 1997 issue of Linux Journal. 

[4] EBCDIC (pronounced "ebb-sid-ick") is an acronym for Extended Binary Coded Decimal Interchange 
Code. This is an IBM data format no longer in much use. A bizarre application of the conv=ebcdic 
option of dd is as a quick 'n easy, but not very secure text file encoder. 


EE 


1 cat $file | dd conv=swab,ebcdic > $file encrypted 

2 4 Encode (looks like gibberish). 

3 4 Might as well switch bytes (swab), too, for a little extra obscurity. 
4 

5 

6 


cat $file encrypted | dd conv=swab,ascii > $file plaintext 
# Decode. 
[5] A macro is a symbolic constant that expands into a command string or a set of operations on 
parameters. Simply put, it's a shortcut or abbreviation. 
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Chapter 17. System and Administrative Commands 


The startup and shutdown scripts in /etc/rc.d illustrate the uses (and usefulness) of many of these 
comands. These are usually invoked by root and used for system maintenance or emergency filesystem 
repairs. Use with caution, as some of these commands may damage your system if misused. 


Users and Groups 


users 
Show all logged on users. This is the approximate equivalent of who -q. 

groups 
Lists the current user and the groups she belongs to. This corresponds to the SGROUPS internal 
variable, but gives the group names, rather than the numbers. 


bash$ groups 
bozita cdrom cdwriter audio xgrp 


bash$ echo $GROUPS 
SOL 


chown, chgrp 
The chown command changes the ownership of a file or files. This command is a useful method that 
root can use to shift file ownership from one user to another. An ordinary user may not change the 
ownership of files, not even her own files. [1] 


root# chown bozo *.txt 


The chgrp command changes the group ownership of a file or files. You must be owner of the 
file(s) as well as a member of the destination group (or root) to use this operation. 


L elegi recursive dunderheads *.data 
2 # The "dunderheads" group will now own all the "*.data" files 
3 #+ all the way down the SEND directory tree (that's what "recursive" means). 
useradd, userdel 
The useradd administrative command adds a user account to the system and creates a home directory 
for that particular user, if so specified. The corresponding userdel command removes a user account 
from the system [2] and deletes associated files. 


8^; The adduser command is a synonym for useradd and is usually a symbolic link to it. 


usermod 
Modify a user account. Changes may be made to the password, group membership, expiration date, 
and other attributes of a given user's account. With this command, a user's password may be locked, 
which has the effect of disabling the account. 

groupmod 
Modify a given group. The group name and/or ID number may be changed using this command. 

id 
The id command lists the real and effective user IDs and the group IDs of the user associated with the 
current process. This is the counterpart to the SUID, SEUID, and $GROUPS internal Bash variables. 


bash$ id 
uid=501 (bozo) gid=501 (bozo) groups=501 (bozo) ,22 (cdrom) , 80 (cdwriter) , 81 (audio) 


bash$ echo SUID 
501 


8^) The id command shows the effective IDs only when they differ from the real ones. 


Also see Example 9-5. 


lid 
The lid (list ID) command shows the group(s) that a given user belongs to, or alternately, the users 
belonging to a given group. May be invoked only by root. 
root# lid bozo 
bozo (gid=500) 
root# lid daemon 
bin (gid=1) 
daemon (gid=2) 
adm (gid=4) 
lp (gid=7) 
who 
Show all users logged on to the system. 
bash$ who 
bozo iis Noe 27 1789 $ 
bozo pts/0 Ape Al 17246 
bozo pts/1 Apr 27 17:47 
bozo pts/2 Aor 27 17:46 
The —m gives detailed information about only the current user. Passing any two arguments to who is 
the equivalent of who -m, as in who am i or who The Man. 
bash$ who -m 
localhost.localdomain!bozo  pts/2 Apr 27 17:49 
whoami is similar to who -m, but only lists the user name. 
bash$ whoami 
bozo 
w 
Show all logged on users and the processes belonging to them. This is an extended version of who. 
The output of w may be piped to grep to find a specific user and/or process. 
bash$ w | grep startx 
Dozo eyi = 4:22pm 6:41 4.47s 0.45s startx 
logname 


Show current user's login name (as found in /var/run/utmp). This is a near-equivalent to 
whoami, above. 


bash$ logname 
bozo 


bash$ whoami 
bozo 


However... 


bash$ su 
PASSO o5sessca 


bash# whoami 
FOOL 
bash# logname 


bozo 


2^) While logname prints the name of the logged in user, whoami gives the name of the 
user attached to the current process. As we have just seen, sometimes these are not the 


same. 

su 
Runs a program or script as a substitute user. su rjones starts a shell as user rjones. A naked su 
defaults to root. See Example A-14. 

sudo 


Runs a command as root (or another user). This may be used in a script, thus permitting a regular 
user to run the script. 


#!/bin/bash 


# Some commands. 
sudo cp /root/secretfile /home/bozo/secret 
5 # Some more commands. 
The file /et c/sudoers holds the names of users permitted to invoke sudo. 
passwd 
Sets, changes, or manages a user's password. 


dL 
2 
3 
4 


The passwd command can be used in a script, but probably should not be. 


Example 17-1. Setting a new password 


! /bin/bash 
setnew-password.sh: For demonstration purposes only. 
Not a good idea to actually run this script. 
UNG SUG must boe runa cus. IE. 


ROOT_UID=0 # Root has SUID O0. 
E WRONG USER-65 # Not root? 


E NOSUCHUSER-70 


SUCCESS=0 
dae | VSI —» TSROOT UTDI] 
then 
echo; echo "Only root can run this script."; echo 
exit $E WRONG USER 
else 
echo 
echo "You should know better than to run this script, root." 


echo "Even root users get the blues... " 


RS IS [RS [RS [S p qp iS [1 ae ie es 
W VBA c9» (er (Ger — ey (b dES GS [em Ae O (Wer es) c3 (ex; (C OS (93. [eS [5 


echo 

Ed 
24 
25 username-bozo 
26 NEWPASSWORD-security violation 
2 
Pas) aie Cige aise Dozo dLabwaexs. Iere, 
29 grep -q "Susername" /etc/passwd 
3 at [ $7 ee SSUCOCCHSS | 
31 then 
32 echo "User Susername does not exist." 


echo "No password changed." 
exit $E NOSUCHUSER 
EL 


w CO ww 
OY CO! Fs W 


ac 


last 


newgrp 


Termin 


tty 


stty 


37 echo "SNEWPASSWORD" | passwd --stdin "Susername" 


38 # The '--stdin' option to 'passwd' permits 

39 #+ getting a new password from stdin (or a pipe). 

40 

41 echo; echo "User $username's password changed!" 

42 

43 # Using the 'passwd' command in a script is dangerous. 
44 

45 exit 0 


The passwd command's -1, —u, and —d options permit locking, unlocking, and deleting a user's 
password. Only root may use these options. 


Show users' logged in time, as read from /var/log/wtmp. This is one of the GNU accounting 
utilities. 


bash$ ac 
total Gis. OS 


List last logged in users, as read from /var/10og/wtmp. This command can also show remote 
logins. 


For example, to show the last few times the system rebooted: 


bash$ last reboot 


reboot system Door 246, RGE ied deo Al 18818 (OW 02) 
reboot system boot 2.6.9-1.667 mex weal 4 i532 (Oil 8272/9 
reboot system boot 2.6.9-1.667 Fri Feb 4 12:56 (00:49) 
reboot syecem boot — 25659-15607 Mow els 3 218019 (O22 E 7/9 


wtmp begins Tue Feb 1 12:50:09 2005 


Change user's group ID without logging out. This permits access to the new group's files. Since users 
may be members of multiple groups simultaneously, this command finds only limited use. 


~) Kurt Glaesemann points out that the newgrp command could prove helpful in setting 
the default group permissions for files a user writes. However, the chgrp command 
might be more convenient for this purpose. 


als 
Echoes the name (filename) of the current user's terminal. Note that each separate xterm window 
counts as a different terminal. 


bash$ tty 
/dev/pts/1 


Shows and/or changes terminal settings. This complex command, used in a script, can control 
terminal behavior and the way output displays. See the info page, and study it carefully. 


Example 17-2. Setting an erase character 


1 #!/bin/bash 
2 4 erase.sh: Using "stty" to set an erase character when reading input. 


3 

4 echo -n "What is your name? " 

5 read name Try to backspace 

6 + to erase characters of input. 

7 Problems? 

8 echo "Your name is $name." 

9 
10 stty erase '#' Set "hashmark" (£4) as erase character. 
11 echo -n "What is your name? " 
12 read name Use 4$ to erase last character typed. 
13 echo "Your name is $name." 
14 
dL «ate. 0) 
16 
17 # Even after the script exits, the new key value remains set. 
18 # Exercise: How would you reset th rase character to the default value? 


Example 17-3. secret password: Turning off terminal echoing 


(XC. 
ec 


cc 
ec 


ec 


ec 


BE 


ec 


(oe) a) fox. (Dub de ©) [Noh |=) (eX wo) TEE (xy (nb des (9X IS) [3 


ec 
19 ec 
20 ec 


22. FNC 


24 ex 


26 # 


read 


ho 


no 


ty 


ho 


read 


ho 
ho 
ho 


ty 


aie 


Do 


#!/bin/bash 
# secret-pw.sh: secret password 


-n "Enter password " 

passwd 

"password is $passwd" 

-n "If someone had been looking over your shoulder, 
"your password would have been compromised." 


&& echo # Two line-feeds in an "and list." 


-echo # Turns off screen echo. 


-n "Enter password again " 
passwd 


"password is $passwd" 


echo # Restores screen echo. 


an 'info stty' for more on this useful-but-tricky command. 


A creative use of stty is detecting a user keypress (without hitting ENTER). 


Example 17-4. Keypress detection 


# on non-GNU systems 


1 #!/bin/bash 

2 4 keypress.sh: Detect a user keypress ("hot keys"). 

3 

4 echo 

5 

6 old tty settings-$(stty -g) # Save old settings (why?) 

T7 gii =—LeaAimom 

8 Keypress-$ (head -c1) iz (ue S (lel lol crowml 2> elem) 
9 


echo 

cho "Key pressed was \""SKeypress"\"." 

echo 

stety "Solo (ES euim # Restore old settings. 


# Thanks, Stephane Chazelas. 


Wer (Oe. c ep) Gi ges (93 [S5 (3 «9 


exec di 


Also see Example 9-3 and Example A-43. 


terminals and modes 


Normally, a terminal works in the canonical mode. When a user hits a key, the resulting character does 
not immediately go to the program actually running in this terminal. A buffer local to the terminal stores 
keystrokes. When the user hits the ENTER key, this sends all the stored keystrokes to the program 
running. There is even a basic line editor inside the terminal. 


bash$ stty -a 
speed 9600 baud; rows 36; columns 96; line = 0; 
intr = ^C; quit = ^N; erase = ^H; kill = ^U; eof = ^D; eol = <undef>; eol2 = «undef»; 
Sitciac = “Of Sic = “Sp deusgo =- “YAS wowing = IRE verase = We lnet = We risi = is 


isig icanon iexten echo echo chok chonl -noflsh -xcase -tostop -echoprt 
Using canonical mode, it is possible to redefine the special keys for the local terminal line editor. 


bash$ cat » filexxx 

wha«ctl-W»I«ctl-H»foo bar«ctl-U»hello world<ENTER> 
«ctl-D» 

bash$ cat filexxx 

hello world 

bash$ wc -c « filexxx 

LZ 


The process controlling the terminal receives only 12 characters (11 alphabetic ones, plus a newline), 
although the user hit 26 keys. 


In non-canonical ("raw") mode, every key hit (including special editing keys such as ctl-H) sends a 
character immediately to the controlling process. 


The Bash prompt disables both icanon and echo, since it replaces the basic terminal line editor with its 
own more elaborate one. For example, when you hit ctl-A at the Bash prompt, there's no ^A echoed by 
the terminal, but Bash gets a M character, interprets it, and moves the cursor to the begining of the line. 


Stéphane Chazelas 


setterm 
Set certain terminal attributes. This command writes to its terminal's st dout a string that changes 
the behavior of that terminal. 


bash$ setterm -cursor off 
bash$ 


The setterm command can be used within a script to change the appearance of text written to 
stdout, although there are certainly better tools available for this purpose. 


setterm -bold on 
echo bold hello 


setterm -bold off 
echo normal hello 


Gus G9 SS ES 


tset 
Show or initialize terminal settings. This is a less capable version of stty. 


bash$ tset -r 
Terminal type is xterm-xfree86. 
KUL as Come colu (AU) . 
limeSerauyoc is Comec col- © (Ca 


setserial 
Set or display serial port parameters. This command must be run by root and is usually found in a 
system setup script. 


# From /etc/pomcia/serial script: 


d 
2 
3 TRO= gertsarial /cew/SDEVICH | sed =e “e/s TROOZ //V^ 
4 setserial /dev/SDEVICE irq 0 ; setserial /dev/SDEVICE irq SIRO 


getty, agetty 
The initialization process for a terminal uses getty or agetty to set it up for login by a user. These 
commands are not used within user shell scripts. Their scripting counterpart is stty. 


mesg 
Enables or disables write access to the current user's terminal. Disabling access would prevent another 
user on the network to write to the terminal. 
į ) It can be quite annoying to have a message about ordering pizza suddenly appear in 
the middle of the text file you are editing. On a multi-user network, you might 
therefore wish to disable write access to your terminal when you need to avoid 
interruptions. 
wall 


This is an acronym for "write all," i.e., sending a message to all users at every terminal logged into the 
network. It is primarily a system administrator's tool, useful, for example, when warning everyone 
that the system will shortly go down due to a problem (see Example 19-1). 


bash$ wall System going down for maintenance in 5 minutes! 
Broadcast mescaga from bozo (qwWos/iLl) Sun Ju. © 13159327 2001... 


System going down for maintenance in 5 minutes! 


$^) If write access to a particular terminal has been disabled with mesg, then wall cannot 
send a message to that terminal. 


Information and Statistics 
uname 
Output system specifications (OS, kernel version, etc.) to stdout. Invoked with the -a option, gives 


verbose system info (see Example 16-5). The —s option shows only the OS type. 


bash$ uname 
Linux 


arch 


bash$ uname -s 
Linux 


bash$ uname -a 


bimi GLC INO“ 2-6. 15-1 2 05/4 1C c rus Mae 14 35848855 nsw 2006 
i686 i686 i386 GNU/Linux 


Show system architecture. Equivalent to uname -m. See Example 11-26. 


bash$ arch 
i686 


bash$ uname -m 
i686 


lastcomm 
Gives information about previous commands, as stored in the /var/account/pacct file. 
Command name and user name can be specified by options. This is one of the GNU accounting 


lastlog 


Isof 


utilities. 


List the last login time of all system users. This references the /var/log/lastlog file. 


bash$ lastlog 

TOON Teyi 
bin 

daemon 


bozo (EE ML 


bash$ lastlog | grep root 


TOOR Tewi, 


mi Dee Y L8gsd9823 0700 20/91 
**Never logged in** 
**Never logged in** 


Sat Dee © Zisi4:829 -0700 2091 


mel Dee 7 19g459221 -0700 2001 


D This command will fail if the user invoking it does not have read permission for the 
/var/log/lastlog file. 


List open files. This command outputs a detailed table of all currently open files and gives 
information about their owner, size, the processes associated with them, and more. Of course, Isof 
may be piped to grep and/or awk to parse and analyze its results. 


bash$ lsof 
COMMAND PTD 
TONE JL 
EISE i 
init JL 
cardmgr 213 


USER FD TYPE DEN MEE Sil vals NODE NAME 

root mem REG 3,5 30748 20303 //eiloxatiau/ abate. 

root mem REG EPI 3 93:2) 8069 /lib/1d-2.1.3.so 
root mem REG 3,5 SILGER (Oris. LUS LSC 21 5 L5 S o9 
root mem REG 355 36956 30357 /sbin/cardmgr 


The Isof command is a useful, if complex administrative tool. If you are unable to dismount a 

filesystem and get an error message that it is still in use, then running /sof helps determine which files 
are still open on that filesystem. The -i option lists open network socket files, and this can help trace 
intrusion or hack attempts. 


bash$ lsof -an -i tcp 


COMMAND PID USER 
ibslacrewt(op« 2/5190) DOZO 
fimekox 2330 bozo 


(HB) AEN EVICE SIZE NODE NAME 
32u IPv4 9956 WE 
38u IPv4 10535 WE 


9 OOo. Os 1LILG c 1L5) D 85 7/5/99 219 1 a JLIL2 5 I, LOA giniwiess 


2 66.0 LIS » 19) 7857 7069) 2:21.6 « 79) «A19 o 2141 BIRETETO) 


strace 


Itrace 


nmap 


nc 


System trace: diagnostic and debugging tool for tracing system calls and signals. This command and 
Itrace, following, are useful for diagnosing why a given program or package fails to run . . . perhaps 
due to missing libraries or related causes. 


bash$ strace df 


exxscwe(/sumaeu. [Verw]; [f a5 vare 1) = © 
uname ({sys="Linux", node="bozo.localdomain", ...}) = 0 
brk (0) = 0x804f5e4 


This is the Linux equivalent of the Solaris truss command. 
Library trace: diagnostic and debugging tool that traces library calls invoked by a given command. 


bash$ ltrace df 
. libc start main(0x804a910, 1, Oxbfb589a4, 0x804fb70, 0x804fb68 «unfinished ...>: 


setlocale(6, "") = "en US.UTF-8" 
IgalimxehierSesie omara (eese aris. Yusr share) locale!) = “sie /sineices/ Loca le’ 
textdomain ("coreutils") = "coreutils" 
__cxa_atexit (0x804b650, 0, 0, 0x8052bf0, Oxbfb58908) = 0 
getenv("DF BLOCK SIZE") = NULL 


Network mapper and port scanner. This command scans a server to locate open ports and the services 
associated with those ports. It can also report information about packet filters and firewalls. This is an 
important security tool for locking down a network against hacking attempts. 


1 #!/bin/bash 

2 

3 SERVER-S$HOST # localhost.localdomain (127.0.0.1). 
4 PORT NUMBER-25 # SMTP port. 

5 

6 nmap SSERVER | grep -w "SPORT NUMBER" # Is that particular port open? 
7 grep -w matches whole words only, 

8 #+ so this wouldn't match port 1025, for example. 

E 
10 exit O0 
JE 
12 s 25/69 open smtp 


The nc (netcat) utility is a complete toolkit for connecting to and listening to TCP and UDP ports. It is 
useful as a diagnostic and testing tool and as a component in simple script-based HTTP clients and 
servers. 


bash$ nc localhost.localdomain 25 
220 localhost.localdomain ESMTP Sendmail 8.13.1/8.13.1; 
Wow, Si Mare 2005 35543895 -0700 


Example 17-5. Checking a remote server for identd 


#! /bin/sh 

## Duplicate DaveG's ident-scan thingie using netcat. Oooh, he'll be p*ssed. 
## Args: target port [port port port ...] 

## Hose stdout _and_ stderr together. 

## 


(nl PES (65. [Sy (55 


# Advantages: runs slower than ident-scan, giving remote inetd less cause 
#+ for alarm, and only hits the few known daemon ports you specify. 

# Disadvantages: requires numeric-only port args, the output sleazitude, 
#+ and won't work for r-services when coming from high source ports. 
Script author: Hobbit <hobbit@avian.org> 

Used in ABS Guide with permission. 


] DARGS=65 # Need at least two args. 
TWO_WINKS=2 # How long to sleep. 

THREE_WINKS=3 

IDPORT=113 # Authentication "tap ident" port. 
RAND1=999 

RAND2=31337 

TIMEOUTO=9 


Sy = Ss Se es I ee 

(S) Wey (es) s (ex; (Oa des eo IS) [—À €» We) (9x p 03 
Gl 
[vv] 
D 


21 TIMEOUT1-8 

22 TIMEOUT2-4 

2S wr 

24 

2:5. (eese S2 u^ qum 

26 un ) enm Wiesel IOS anrd cur Laast Cine POR." 2° exit Sit_VNDARGS 93 
27 esac 

28 

2:9 Ping 'em once and s if they *are* running identd. 
30 nc -z -w STIMEOUTO "$1" SIDPORT || \ 

Si 4| exeo “ojos, Sil sbeua ir suima 3hgkewegl." p esate © 2 } 
92 -z scans for listening daemons. 

315 -w STIMEOUT = How long to try to connect. 

34 

39 Generate a randomish base port. 

36 RP- expr $$ $ SRAND1 + SRAND2~ 

Si 

38 TRG="S1i" 

SS) gimitiE 

40 

4L Wise test US" p» clo 


42 me —- cx STIMETOUTIL =o STRE) Wemsg STIL < feti > /cleny/inbulil & 
43 PROC-S! 

44 Sleep STHREE WINKS 

45 echo "S(1),S(RP)" | nc -w STIMEOUT2 -r "STRG" SIDPORT 2»&1 
46 Sleep $TWO WINKS 

47 

48 # Does this look like a lamer script or what . . . ? 

49 # ABS Guide author comments: "Ain't really all that bad 
50 #+ kinda clever, actually." 
5i 

52 PaL =RU SPROG 

155) RP= expr S{RP} + 1^ 

54 shift 

55 done 

56 

57 ewig Gm? 

58 

3 Notes 

QU te m 

61 

62 Try commenting out line 30 and running this script 
63 #+ with "localhost.localdomain 25" as arguments. 

64 

65 For more of Hobbit's 'nc' example scripts, 

66 #+ look in the documentation: 

67 #+ the /usr/share/doc/nc-X.XX/scripts directory. 


And, of course, there's Dr. Andrew Tridgell's notorious one-line script in the BitKeeper Affair: 


1 echo clone | nc thunk.org 5000 > e2fsprogs.dat 


free 
Shows memory and cache usage in tabular form. The output of this command lends itself to parsing, 
using grep, awk or Perl. The procinfo command shows all the information that free does, and much 
more. 
bash$ free 
Total used ETES shared Durers cached 
Mem: 30504 28624 1880 159210 1608 16376 
-/* buffers/cache: 10640 19864 
Swap: 68540 3128 65412 
To show unused RAM memory: 
bash$ free | grep Mem | awk '{ print $4 }' 
1880 
procinfo 
Extract and list information and statistics from the /proc pseudo-filesystem. This gives a very 
extensive and detailed listing. 
bash$ procinfo | grep Bootup 
iEoxowewjos Meel Mare 21 dogi5s50 2001. Load average: 0.04 0.21 0.34 3/47 6829 
Isdev 
List devices, that 1s, show installed hardware. 
bash$ lsdev 
Device DMA iO 1/0 Pores 
cascade 4 2 
dma 0080-008£f 
dmal 0000-001f 
dma2 00c0-00df 
fpu 00£0-00£ff 
ided LA — (9)3bxE(9)—(0)1538 7 O) Sue GO) 9 385 
du 
Show (disk) file usage, recursively. Defaults to current working directory, unless otherwise specified. 
bash$ du -ach 
1.0k ./wi.sh 
1.0k a EIE. o em 
ilk ./random.file 
6.0k 6 
6.0k total 
df 
Shows filesystem usage in tabular form. 
bash$ df 
Filesystem Iie LOCIES Used Available Use% Mounted on 
/dev/hda5 2792/62 92607 166547 36$ / 
/ dev/hda8 222525 L2: 3 9)51L 87085 59$ /home 
/ dev/hda7 1408796 1075744 261488 80% /usr 
dmesg 


Lists all system bootup messages to st dout. Handy for debugging and ascertaining which device 
drivers were installed and which system interrupts in use. The output of dmesg may, of course, be 
parsed with grep, sed, or awk from within a script. 


bash$ dmesg | grep hda 
Kernel command line: ro root-/dev/hda2 


hda: IBM-DLGA-23080, ATA DISK drive 
hda: 6015744 sectors (3080 MB) w/96KiB Cache, CHS-746/128/63 
hda: hdal hda2 hda3 « hda5 hda6 hda7 > hda4 


stat 
Gives detailed and verbose statistics on a given file (even a directory or device file) or set of files. 
bash$ stat test.cru 
Palle Wires sere! 
Size: 49970 Allocated Blocks: 100 Filetype: Regular File 
Mode: (0664/-rw-rw-r--) uicis ( SiOil/ bozo Gic ( — S917 bozo) 
Device: 3,8 Inode: 18185 Tabakas d 
ACCESS Se Gui 2 5240824: (19 
Modify: Sat Jun 2 16:40:24 2001 
Change: Sat Jun 2 16:40:24 2001 
If the target file does not exist, stat returns an error message. 
bash$ stat nonexistent-file 
nonexistent-file: No such file or directory 
In a script, you can use stat to extract information about files (and filesystems) and set variables 
accordingly. 
1 #!/bin/bash 
2 # fileinfo2.sh 
3 
4 # Per suggestion of Joél Bourquard and : 
5 4 http://www.linuxquestions.org/questions/showthread.php?t-410766 
6 
7 
8 FILENAME-testfile.txt 
9 file name-$(stat -c$n "SFILENAME") i Same as "SFILENAME" of course. 
10 file owner-$(stat -c$U "SFILENAME") 
11 file size-$(stat -c$s "SFILENAME") 
12 # Certainly easier than using "ls -1 SFILENAME" 
13 #+ and then parsing with sed. 
14 file inode-$(stat -c$i "SFILENAME") 
15 file type-$(stat -c$F "SFILENAME") 
16 file access rights-$(stat -c$A "SFILENAME") 
alley 
LÌ echo Vialile: mans: Sita le meme” 
19 echo "File owner: Sfile_owner" 
20 cimo; Vgl coles Saule Size” 
2iL @elac “ia. ile alingyeles Sfile_inode" 
22 echo "File type: $file type" 
23 echo "File access rights: $file access rights" 
24 
25 exit 0 
26 
By gl iex Meanie «o: 
28 
29 File name: testfile.txt 
30 File owner: bozo 
SUL mile üben 418 
32 File inode: i7 3013 7$ 
33 File type: regular file 
Say TNS ACCESS ALC 5 NEISWES WEISSE 
vmstat 


Display virtual memory statistics. 


bash$ vmstat 
procs memory swap io system cpu 


E o de gw swpd free buff cache si so Iit bo in GS ws sw iol 
0 0 0 11040 2536 939952 0 0 38 7 Aya 88 8 3 BS 


netstat 
Show current network statistics and information, such as routing tables and active connections. This 
utility accesses information in /proc/net (Chapter 29). See Example 29-4. 


netstat -r is equivalent to route. 


bash$ netstat 
Active Internet connections (w/o servers) 


Proto Recv-Q Send-Q Local Address Foreign Address State 
Active UNIX domain sockets (w/o servers) 

Proto RefCnt Flags Type State I-Node Path 

tims Lal I J DGRAM 906 /dev/log 

Winans 3B iT STREAM CONNECTED 4514 Hempstead x0 
unix 3 [ ] STREAM CONNECTED 4513 


8^; A netstat -Iptu shows sockets that are listening to ports, and the associated processes. 
This can be useful for determining whether a computer has been hacked or 
compromised. 
uptime 
Shows how long the system has been running, along with associated statistics. 


bash$ uptime 
IO gen duo ESN, 3 users, load average: 0.17, 0.34, 0.27 
8^) A load average of 1 or less indicates that the system handles processes immediately. A 
load average greater than 1 means that processes are being queued. When the load 
average gets above 3, then system performance is significantly degraded. 
hostname 
Lists the system's host name. This command sets the host name in an /etc/rc.d setup script 
(/etc/rc.d/rc.sysinit or similar). It is equivalent to uname -n, and a counterpart to the 
SHOSTNAME internal variable. 


bash$ hostname 
localhost.localdomain 


bash$ echo $HOSTNAME 
localhost.localdomain 


Similar to the hostname command are the domainname, dnsdomainname, nisdomainname, and 
ypdomainname commands. Use these to display or set the system DNS or NIS/YP domain name. 
Various options to hostname also perform these functions. 


hostid 
Echo a 32-bit hexadecimal numerical identifier for the host machine. 


bash$ hostid 
7£0100 
8^) This command allegedly fetches a "unique" serial number for a particular system. 
Certain product registration procedures use this number to brand a particular user 
license. Unfortunately, hostid only returns the machine network address in 
hexadecimal, with pairs of bytes transposed. 


The network address of a typical non-networked Linux machine, is found in 
/etc/hosts. 


bash$ cat /etc/hosts 


L2 7 50). 3L localhost.localdomain localhost 
As it happens, transposing the bytes of 127.0.0.1, we get 0.127.1.0, which 
translates in hex to 007£0100, the exact equivalent of what hostid returns, above. 
There exist only a few million other Linux machines with this identical hostid. 


sar 
Invoking sar (System Activity Reporter) gives a very detailed rundown on system statistics. The 
Santa Cruz Operation ("Old" SCO) released sar as Open Source in June, 1999. 
This command is not part of the base Linux distribution, but may be obtained as part of the sysstat 
utilities package, written by Sebastien Godard. 
bash$ sar 
Limus 249 (brooke Seringas. Er) 09/26/03 
10230 30 CPU $user $nice Ssystem Siowait Sidle 
10:40:00 all ZH 10), QO) 65.48 0.00 ZIAN 
1E(9) 8 SO) 8 OO) all 34 35 0.00 12.5.35 0.00 24.28 
il 8:900) 8 (00) all ibo dE 0.00 10) 4 0/7) 0.00 JL) o ILL 
Average: all 254.29 3563 12.5687 0.00 2L. 5271 
Al S92 s (630) LINUX RESTART 
15200200 CPU $user $nice $system $iowait Sidle 
155 10-00 aLi Sas 2.40 JU Say) 0.00 Ti s d 
1520:00 all 4.07 1.00 JL3L 5 BS 0.00 9/2 5 BS 
1.5.8 30 8 (09) all omo 2s 94 74 55 0.00 99) c. T1 
Average: all 6.33 Ta 70 14.71 0.00 114216 
readelf 
Show information and statistics about a designated e/f binary. This is part of the binutils package. 
bash$ readelf -h /bin/bash 
ELF Header: 
Magic: quz die le 46 Oi O1 OL OO 00 OO 00 OH OO OO WO OD 
Classes ELF32 
Data: 2's complement, little endian 
Version: 1 (current) 
OS/ABI: UNIX - System V 
ABI Version: 0 
Type: EXEC (Executable file) 
size 
The size [/path/to/binary] command gives the segment sizes of a binary executable or archive file. 
This is mainly of use to programmers. 
bash$ size /bin/bash 
text data bss dec hex filename 
ASIS SNA 22496 173992 5959859 82433 /bin/bash 
System Logs 
logger 


Appends a user-generated message to the system log (/var/log/messages). You do not have to 
be root to invoke logger. 


1 logger Experiencing instability in network connection at 23:10, 05/21. 
2 4 Now, do a 'tail /var/log/messages'. 


By embedding a logger command in a script, it is possible to write debugging information to 


/var/log/messages. 


logger -t $0 -i Logging at line "SLINENO". 
# The "-t" option specifies the tag for the 
# The "-i" option records the process ID. 


# tail /var/log/message 
# 


xu ep) (nl dE (es) [ES [5 


logrotate 


This utility manages the system log files, rotating, compres 


5 ud. 7 20:47:58 localhost ies sm| 7121 2 


logger entry. 


Logging at line 3. 


sing, deleting, and/or e-mailing them, as 


appropriate. This keeps the / var /1og from getting cluttered with old log files. Usually cron runs 


logrotate on a daily basis. 


Adding an appropriate entry to /etc/logrotate.coní 
files, as well as system-wide ones. 


f makes it possible to manage personal log 


$^) Stefano Falsetto has created rottlog, which he considers to be an improved version of 


logrotate. 
Job Control 
ps 
Process Statistics: lists currently executing processes by owner and PID (process ID). This is usually 
invoked with ax or aux options, and may be piped to grep or sed to search for a specific process (see 
Example 15-14 and Example 29-3). 
bash$ ps ax | grep sendmail 
295 ? S 0:00 sendmail: accepting connections on port 25 
To display system processes in graphical "tree" format: ps afjx or ps ax --forest. 
pgrep, pkill 
Combining the ps command with grep or kill. 
bash$ ps a | grep mingetty 
2212 i32 Sst 0:00 /sbin/mingetty tty2 
221.9 EEVI SSF 0:00 /sbin/mingetty tty3 
2214 tty4 Sst 0:00 /sbin/mingetty tty4 
22/15, CEVI SSH 0:00 /sbin/mingetty tty5 
22115 i36 Sst 0:00 /sbin/mingetty tty6 
4849 pts/2 S+ 0:00 grep mingetty 
bash$ pgrep mingetty 
2212 mingetty 
2213 mingetty 
2214 mingetty 
2215 mingetty 
2216 mingetty 
Compare the action of pkill with killall. 
pstree 
Lists currently executing processes in "tree" format. The —p option shows the PIDs, as well as the 
process names. 
top 


Continuously updated display of most cpu-intensive processes. The -b option displays in text mode, 


so that the output may be parsed or accessed from a script. 


nice 


nohup 


pidof 


bash$ top -b 
SssViom tis S mim, © Users,  loescl eweracges 04.49. 0.92. (0.139 
45 processes: 44 sleeping, 1 running, 0 zombie, 0 stopped 
CPU states: 13.6$ user, 7.99 System, 0.0% nice, 78.9% idle 


Mem: 78396K av, 65468K used, 12928K free, OK shrd, 2352K buff 
Swap:  157208K av, OK used, 157208K free 37244K cached 
PID USER PRI TNI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 
848 bozo iy 0 996 J96 800 R 366 eZ 0:00 tem 
l zo 8 0 5412 S12 444 S Q9 W.6 (gg umi 
Z TX 9 0 0 0 0 SW 01.0) 0.0 0:00 keventd 


Run a background job with an altered priority. Priorities run from 19 (lowest) to -20 (highest). Only 
root may set the negative (higher) priorities. Related commands are renice and snice, which change 
the priority of a running process or processes, and skill, which sends a kill signal to a process or 
processes. 


Keeps a command running even after user logs off. The command will run as a foreground process 
unless followed by &. If you use nohup within a script, consider coupling it with a wait to avoid 
creating an orphan or zombie process. 


Identifies process ID (PID) of a running job. Since job control commands, such as kill and renice act 
on the PID of a process (not its name), it is sometimes necessary to identify that PID. The pidof 
command is the approximate counterpart to the $PPID internal variable. 


bash$ pidof xclock 
880 


Example 17-6. pidof helps kill a process 


1 #!/bin/bash 
2 # kill-process.sh 
E 
4 NOPROCESS-2 
5 
6 process-xxxyyyzzz # Use nonexistent process. 
7 For demo purposes only... 
8 Glow i wane EO acirwadlily kO eu excel process woen tals Sere. 
9 
10 If, for example, you wanted to use this script to logoff the Internet, 
ALTE process-pppd 
1,2 
13 t-'pidof S$process' # Find pid (process id) of Sprocess. 
14 The pid is needed by 'kill' (can't 'kill' by program name). 
15 
iG ai [| —z “Si” | # If process not present, 'pidof' returns null. 
17 then 
18 echo "Process Sprocess was not running." 
LS echo "Nothing killed." 
20 exit SNOPROCESS 
21 363 
22 
2,3 ILLE Sie # May need 'kill -9' for stubborn process. 
24 


25 4 Need a check here to s if process allowed itself to be killed. 
26 4 Perhaps another " t= pidof S$process' " or 


fuser 


cron 


27 


28 

29 # This entire script could be replaced by 
30 # kill $(pidof -x process name) 

Sul wp (xe 

32 s killall process name 

33 4 but it would not be as instructive. 

34 

35 ew 0 


Identifies the processes (by PID) that are accessing a given file, set of files, or directory. May also be 
invoked with the -k option, which kills those processes. This has interesting implications for system 
security, especially in scripts preventing unauthorized users from accessing system services. 


bash$ fuser -u /usr/bin/vim 
Jie Aor STER 3207e (bozo) 


bash$ fuser -u /dev/null 
/dev/null: 300950250) 301045029) 219i (bozo) 3190 ( bozo) 


One important application for fuser is when physically inserting or removing storage media, such as 
CD ROM disks or USB flash drives. Sometimes trying a umount fails with a device is busy error 
message. This means that some user(s) and/or process(es) are accessing the device. An fuser -um 
/dev/device_name will clear up the mystery, so you can kill any relevant processes. 


bash$ umount /mnt/usbdrive 
umount: /mnt/usbdrive: device is busy 


bash$ fuser -um /dev/usbdrive 
/mnt/usbdrive: MZ ooo) 


bash$ kill -9 1772 
bash$ umount /mnt/usbdrive 


The fuser command, invoked with the -n option identifies the processes accessing a port. This is 
especially useful in combination with nmap. 


root# nmap localhost.localdomain 
PORT STATE SERVICE 
2S ECD open smtp 


root# fuser -un tcp 25 
2S EEPE 2095 (root) 


root# ps ax | grep 2095 | grep -v grep 
2095 ? Ss 0:00 sendmail: accepting connections 


Administrative program scheduler, performing such duties as cleaning up and deleting system log 
files and updating the slocate database. This is the superuser version of at (although each user may 
have their own crontab file which can be changed with the crontab command). It runs as a daemon 
and executes scheduled entries from /etc/crontab. 


$") Some flavors of Linux run crond, Matthew Dillon's version of cron. 


Process Control and Booting 


init 
The init command is the parent of all processes. Called in the final step of a bootup, init determines 
the runlevel of the system from /etc/inittab. Invoked by its alias telinit, and by root only. 
telinit 
Symlinked to init, this is a means of changing the system runlevel, usually done for system 
maintenance or emergency filesystem repairs. Invoked only by root. This command can be dangerous 
-- be certain you understand it well before using! 
runlevel 


Shows the current and last runlevel, that is, whether the system is halted (runlevel 0), in single-user 
mode (1), in multi-user mode (2 or 3), in X Windows (5), or rebooting (6). This command accesses 
the /var/run/utmp file. 

halt, shutdown, reboot 
Command set to shut the system down, usually just prior to a power down. 


On some Linux distros, the halt command has 755 permissions, so it can be invoked 
by a non-root user. A careless halt in a terminal or a script may shut down the system! 
service 
Starts or stops a system service. The startup scripts in /etc/init.dand /etc/rc.d use this 
command to start services at bootup. 


root# /sbin/service iptables stop 


Flushing firewall rules: [ OR | 

Setting chains to policy ACCEPT: filter L Qu 1 

Unloading iptables modules: L Ox | 
Network 


ifconfig 
Network interface configuration and tuning utility. 


bash$ ifconfig -a 

lo Link encap:Local Loopback 
3umeiE aclelegil2Z7 0.0.1 Measles 255.0.0.0 
UP LOOPBACK RUNNING MTU:16436 Metric:1 
RX packets:10 errors:0 dropped:0 overruns:0 frame:0 
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:0 
RX bytes:700 (700.0 b) TX bytes:700 (700.0 b) 


The ifconfig command is most often used at bootup to set up the interfaces, or to shut them down 
when rebooting. 


Code snippets from /etc/rc.d/init.d/network 


Check that networking is up. 
S(NETWORKING) = "no" ] && exit 0 


=x d'myouew/stitexoweuraber | || || esit 0 


O We) (ee) | en) Gil des te) [ey eS 


pas 


JUL 

L2 3&gue iL aia Siisieeuciraeceas B clo 

13 if ifconfig $i 2>/dev/null | grep -q "UP" >/dev/null 2>&1 ; then 

14 action "Shutting down interface $i: " ./ifdown $i boot 

15 iE al 

16 $ The GNU-specific "-q" option to "grep" means "quiet", i.e., 

17 #+ producing no output. 

18 # Redirecting output to /dev/null is therefore not strictly necessary. 
1$ 


20 # 

2 

22 echo "Currently active devices:" 

25) echo ~ /sloalin/iiecomirie || cee eszi || aw: "dps Sp" 

24 # ^^^^^ should be quoted to prevent globbing. 
25 # The following also work. 

26 4 ceha S(U/eloxswuieemsie; | awk U"J/^le-2z]4/ 4 prine Sd n) 

27 4 echo S(/sius/iiesmxe | med -e "s/f *//") 

Ae s wes. S.C., wow aCe erona (omes s 


See also Example 31-6. 


iwconfig 


ip 


This is the command set for configuring a wireless network. It is the wireless equivalent of ifconfig, 


above. 


General purpose utility for setting up, changing, and analyzing /P (Internet Protocol) networks and 


attached devices. This command is part of the iproute2 package. 


bash$ ip link show 
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue 
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo fast qlen 1000 
JL exiens WOScOsHSseesee gels: Jewel Iie Girie siete Babe gati Brie 
3: sit0: <NOARP> mtu 1480 qdisc noop 
lack /site 0-0-0-0 brda 0.0.0.0 


bash$ ip route list 
169.254.0.0/16 dev lo scope link 


Or, in a script: 


!/bin/bash 
Serio dy diuo. Nicolas Ruiz 
Used with his kind permission. 


Setting up (and stopping) a GRE tunnel. 


Skart E limited c reor 


OCA drew 1L8)2 WOE) db o LH 
REMOTES P= MOMOM SEI W 
OMHERSTEACHE SA O LOS. 10) Ow 
REMOTE NET-"192.168.3.0/24" 


H m oH 
OY O' 4& Q). I9. IB. O (o O0 -1 O Or i8 0) ND S 


e 


e 


/sbin/ip tunnel add netb mode gre remote S$REMOTE IP \ 
local SiO Euh ma Tell 255 

/sbin/ip addr add $OTHER IFACE dev netb 

/sbin/ip link set netb up 

/sbin/ip route add $REMOTE NET dev netb 


Nere 
€» «p oN 


exit O 4AE IE AE IE tE AE AE HE AE IE AE IE FE AE FE HE AE FE AE IE FE AE FE FE AE FE AE IE FE AE FE FE AE FE AE HE FE AE HE HE H 


NNN 
w Ne 
+ 


stop-tunnel.sh 


24 

25 REMOTE NET-"192.168.3.0/24" 

26 

27 /sbin/ip route del S$REMOTE NET dev netb 
28 /sbin/ip link set netb down 

29 /sbin/ip tunnel del netb 


30 
31 exit 0 
route 
Show info about or make changes to the kernel routing table. 
bash$ route 
Destination Gateway Genmask Flags MSS Window  irtt Iface 
jeun3-6 7 .Jleomesieo * 2554255425549 515. Wil 40 0 0 pppO 
32/7/5109 5 0)5 (9 * 295605050 U 40 0 0) le 
default jouas- 65 7 loozosusia 0.0.0.0 UG 40 0 0 pppO 
chkconfig 
Check network and system configuration. This command lists and manages the network and system 
services started at bootup in the /et c/rc?.d directory. 
Originally a port from IRIX to Red Hat Linux, chkconfig may not be part of the core installation of 
some Linux flavors. 
bash$ chkconfig --list 
atd (0) (OVE 3E 1L rox 3E 2 E OVE IE S RO 4:on 5:on 6:off 
rwhod ORO E IL BOVE RE 2 B UE IE SIBIGUEIE 4A:off SRON E GROTE 
tcpdump 
Network packet "sniffer." This is a tool for analyzing and troubleshooting traffic on a network by 
dumping packet headers that match specified criteria. 
Dump ip packet traffic between hosts bozoville and caduceus: 
bash$ tcpdump ip host bozoville and caduceus 
Of course, the output of tepdump can be parsed with certain of the previously discussed text 
processing utilities. 
Filesystem 
mount 


Mount a filesystem, usually on an external device, such as a floppy or CDROM. The file 
/etc/fstab provides a handy listing of available filesystems, partitions, and devices, including 
options, that may be automatically or manually mounted. The file /etc/mtab shows the currently 
mounted filesystems and partitions (including the virtual ones, such as /proc). 


mount -a mounts all filesystems and partitions listed in /etc/fstab, except those with a noauto 
option. At bootup, a startup script in /etc/rc.d(rc.sysinit or something similar) invokes this 
to get everything mounted. 


1 mount -t iso9660 /dev/cdrom /mnt/cdrom 

2 4 Mounts CD ROM. ISO 9660 is a standard CD ROM filesystem. 
3 mount /mnt/cdrom 

4| s» Shorten, wie /inie/echeom laistescl ain /erte/ taras 


The versatile mount command can even mount an ordinary file on a block device, and the file will act 
as if it were a filesystem. Mount accomplishes that by associating the file with a loopback device. One 
application of this is to mount and examine an ISO9660 filesystem image before burning it onto a 
CDR. [3] 


Example 17-7. Checking a CD image 


# As root... 


mkdir /mnt/cdtest 4 Prepare a mount point, if not already there. 


# uo loopt option cobayaileme ice Ylosetuo /clew/ loog 
cd /mnt/cdtest # Now, check the image. 
lg -alr # List the files in the directory tree there. 


al 
2 
3 
4 
5 mortme -r =e 1600660 - loop ecl-inacge.isie /mime/echoesic # Mount the image. 
6 
j 
8 
9 # And so forth. 


umount 


gnome- 


sync 


losetup 


Unmount a currently mounted filesystem. Before physically removing a previously mounted floppy or 
CDROM disk, the device must be umounted, else filesystem corruption may result. 


1 umount /mnt/cdrom 
2 4 You may now press the eject button and safely remove the disk. 


8^; The automount utility, if properly installed, can mount and unmount floppies or 
CDROM disks as they are accessed or removed. On "multispindle" laptops with 
swappable floppy and optical drives, this can cause problems, however. 
mount 
The newer Linux distros have deprecated mount and umount. The successor, for command-line 
mounting of removable storage devices, is gnome-mount. It can take the -d option to mount a device 
file by its listing in /dev. 


For example, to mount a USB flash drive: 


bash$ gnome-mount -d /dev/sdal 
gnome-mount 0.4 


bash$ df 


/ dev/sdal 63584 12034 51550 19$ /media/disk 


Forces an immediate write of all updated data from buffers to hard drive (synchronize drive with 
buffers). While not strictly necessary, a sync assures the sys admin or user that the data just changed 
will survive a sudden power failure. In the olden days, a sync; sync (twice, just to make 
absolutely sure) was a useful precautionary measure before a system reboot. 


At times, you may wish to force an immediate buffer flush, as when securely deleting a file (see 
Example 16-60) or when the lights begin to flicker. 


Sets up and configures loopback devices. 


Example 17-8. Creating a filesystem in a file 


1 SIZE-1000000 # 1 meg 

2 

3 head -c SSIZE « /dev/zero > file # Set up file of designated siz 
4 losetup /dev/loopO file # Set it up as loopback device. 
5 mke2fs /dev/loopO # Create filesystem. 

6 mount -o loop /dev/loopO /mnt f Mount it. 

7 

G s hanks SoC. 


mkswap 

Creates a swap partition or file. The swap area must subsequently be enabled with swapon. 
swapon, swapoff 

Enable / disable swap partitition or file. These commands usually take effect at bootup and shutdown. 
mke2fs 

Create a Linux ext2 filesystem. This command must be invoked as root. 


Example 17-9. Adding a new hard drive 


!/bin/bash 


Adding a second hard drive to system. 

Software configuration. Assumes hardware already mounted. 
From an article by the author of the ABS Guide. 

In issue #38 of Linux Gazette , http://www. linuxgazette.com. 


ROOT UID-0 # This script must be run as root. 
E NOTROOT-67 # Non-root exit error. 


ie [[ SUID? -me VOROOT uU T 

then 
Seine Yast be root cO rin Cg Semi. Uu 
exit $E NOTROOT 

tea 


# Use with extreme caution! 
# If something goes wrong, you may wipe out your current filesystem. 


NPRPRPRP PPP PP 
O (o 0» -1 O Ui i& (). NM P. O xo 0 -1 O UI 4 Q I E 


21 NEWDISK-/dev/hdb # Assumes /dev/hdb vacant. Check! 
22 MOUNTPOINT-/mnt/newdisk 4 Or choose another mount point. 
23 

24 

25 fdisk SNEWDISK 


26 mke2fs -cv SNEWDISK1 # Check for bad blocks (verbose output). 
27 # Note: 2. /dev/hdb1, *not* /dev/hdb! 

28 mkdir SMOUNTPOINT 

29 chmod 777 $MOUNTPOINT # Makes new drive accessible to all users. 
30 

31 

Je Now, test 

33 mount -t ext2 /dev/hdbl /mnt/newdisk 

34 Try creating a directory. 

35 If it works, umount it, and proceed. 

36 

37 Final step: 

38 Add the following line to /etc/fstab. 

39 /dev/hdbl /mnt/newdisk ext2 defaults 1 1 

40 

41 exit 


See also Example 17-8 and Example 30-3. 


tune2fs 


Tune ext2 filesystem. May be used to change filesystem parameters, such as maximum mount count. 
This must be invoked as root. 


This is an extremely dangerous command. Use it at your own risk, as you may 
inadvertently destroy your filesystem. 


dumpe2fs 


Dump (list to st dout) very verbose filesystem info. This must be invoked as root. 


root# dumpe2fs /dev/hda7 | grep 'ount count' 

gluwse2is 1.19, i 3-gmi-2000) io mxr2 mS 0.5. 95/06/09 
Mount count: 6 

Maximum mount count: 20 


hdparm 


fdisk 


List or change hard disk parameters. This command must be invoked as root, and it may be dangerous 
if misused. 


Create or change a partition table on a storage device, usually a hard drive. This command must be 
invoked as root. 


Use this command with extreme caution. If something goes wrong, you may destroy 
an existing filesystem. 


fsck, e2fsck, debugfs 


Filesystem check, repair, and debug command set. 


fsck: a front end for checking a UNIX filesystem (may invoke other utilities). The actual filesystem 
type generally defaults to ext2. 


e2fsck: ext2 filesystem checker. 


debugfs: ext2 filesystem debugger. One of the uses of this versatile, but dangerous command is to 
(attempt to) recover deleted files. For advanced users only! 


All of these should be invoked as root, and they can damage or destroy a filesystem if 
misused. 


badblocks 


Checks for bad blocks (physical media flaws) on a storage device. This command finds use when 
formatting a newly installed hard drive or testing the integrity of backup media. [4] As an example, 
badblocks /dev/fd0 tests a floppy disk. 


The badblocks command may be invoked destructively (overwrite all data) or in non-destructive 
read-only mode. If root user owns the device to be tested, as is generally the case, then root must 
invoke this command. 


Isusb, usbmodules 


The Isusb command lists all USB (Universal Serial Bus) buses and the devices hooked up to them. 
The usbmodules command outputs information about the driver modules for connected USB devices. 


bash$ lsusb 
Bus 001 Device 001: ID 0000:0000 
Device Descriptor: 


bLength JL) 
bDescriptorType i 
bcdUSB 1.00 


bDeviceClass 9 Hub 


bDeviceSubClass 0 


bDeviceProtocol 0 
bMaxPacketSize0 8 
idVendor 0x0000 
idProduct 0x0000 


Ispci 
Lists pci busses present. 


bash$ lspci 
00:00.0 Host bridge: Intel Corporation 82845 845 
(Brookdale) Chipset Host Bridge (rev 04) 
00:01.0 PCI bridge: Intel Corporation 82845 845 
(Brookdale) Chipset AGP Bridge (rev 04) 
00:1d.0 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #1) (rev 02) 
00:1d.1 USB Controller: Intel Corporation 82801CA/CAM USB (Hub 42) (rev 02) 
00:1d.2 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #3) (rev 02) 
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 42) 


mkbootdisk 
Creates a boot floppy which can be used to bring up the system if, for example, the MBR (master boot 
record) becomes corrupted. Of special interest is the -—iso option, which uses mkisofs to create a 
bootable /5$09660 filesystem image suitable for burning a bootable CDR. 


The mkbootdisk command is actually a Bash script, written by Erik Troan, in the / sbin directory. 
mkisofs 
Creates an [SO9660 filesystem suitable for a CDR image. 
chroot 
CHange ROOT directory. Normally commands are fetched from $PATH, relative to /, the default 
root directory. 'This changes the root directory to a different one (and also changes the working 
directory to there). This is useful for security purposes, for instance when the system administrator 
wishes to restrict certain users, such as those telnetting in, to a secured portion of the filesystem (this 
is sometimes referred to as confining a guest user to a "chroot jail"). Note that after a chroot, the 
execution path for system binaries is no longer valid. 


A chroot /opt would cause references to /usr/bin to be translated to /opt /usr/bin. 
Likewise, chroot /aaa/bbb /bin/1s would redirect future instances of Is to /aaa/bbb as 
the base directory, rather than / as is normally the case. An alias XX 'chroot /aaa/bbb Is' in a user's 
^ / .bashrc effectively restricts which portion of the filesystem she may run command "XX" on. 


The chroot command is also handy when running from an emergency boot floppy (chroot to 

/ dev/ £d0), or as an option to lilo when recovering from a system crash. Other uses include 
installation from a different filesystem (an rpm option) or running a readonly filesystem from a CD 
ROM. Invoke only as root, and use with care. 


D It might be necessary to copy certain system files to a chrooted directory, since the 
normal $PATH can no longer be relied upon. 
lockfile 


This utility is part of the procmail package (www.procmail.org). It creates a lock file, a semaphore 
that controls access to a file, device, or resource. 


Definition: A semaphore is a flag or signal. (The usage originated in railroading, where a 
colored flag, lantern, or striped movable arm semaphore indicated whether a particular track was in 
use and therefore unavailable for another train.) A UNIX process can check the appropriate 
semaphore to determine whether a particular resource is available/accessible. 


The lock file serves as a flag that this particular file, device, or resource is in use by a process (and is 
therefore "busy"). The presence of a lock file permits only restricted access (or no access) to other 
processes. 


lockfile /home/bozo/lockfiles/$0.lock 
# Creates a write-protected lockfile prefixed with the name of the script. 


lockfile /home/bozo/lockfiles/S[(044*/]).l1ock 
5 # A safer version of the above, as pointed out by E. Choroba. 


Lock files are used in such applications as protecting system mail folders from simultaneously being 
changed by multiple users, indicating that a modem port is being accessed, and showing that an 
instance of Firefox is using its cache. Scripts may check for the existence of a lock file created by a 
certain process to check if that process is running. Note that if a script attempts to create a lock file 
that already exists, the script will likely hang. 


Normally, applications create and check for lock files in the /var/1ock directory. [5] A script can 
test for the presence of a lock file by something like the following. 


1 appname-xyzip 
2 q Application Vs" createed lock file V /vaie/ xl 219. lockt, 
9 
4 if [ -e "/var/lock/Sappname.lock" ] 
5 then #+ Prevent other programs & scripts 
6 # from accessing files/resources used by xyzip. 
3 
flock 
Much less useful than the lockfile command is flock. It sets an "advisory" lock on a file and then 
executes a command while the lock is on. This is to prevent any other process from setting a lock on 
that file until completion of the specified command. 
L fex! (S) cat $0 > lockiile S0 
2 # Set a lock on the script the above line appears in, 
3 #+ while listing the script to stdout. 
ə Unlike lockfile, flock does not automatically create a lock file. 
mknod 


Creates block or character device files (may be necessary when installing new hardware on the 
system). The MAKEDEYV utility has virtually all of the functionality of mknod, and is easier to use. 
MAKEDEV 
Utility for creating device files. It must be run as root, and in the /dev directory. It is a sort of 
advanced version of mknod. 
tmpwatch 
Automatically deletes files which have not been accessed within a specified period of time. Usually 
invoked by cron to remove stale log files. 


Backup 


dump, restore 
The dump command is an elaborate filesystem backup utility, generally used on larger installations 
and networks. [6] It reads raw disk partitions and writes a backup file in a binary format. Files to be 
backed up may be saved to a variety of storage media, including disks and tape drives. The restore 
command restores backups made with dump. 


fdformat 


Perform a low-level format on a floppy disk (/ dev/ £d0 *). 


System Resources 


ulimit 
Sets an upper limit on use of system resources. Usually invoked with the —£ option, which sets a limit 
on file size (ulimit -f 1000 limits files to 1 meg maximum). [7] The -t option limits the coredump 
size (ulimit -c 0 eliminates coredumps). Normally, the value of ulimit would be set in 
/etc/profile and/or -/.bash profile (see Appendix G). 
Q Judicious use of ulimit can protect a system against the dreaded fork bomb. 
1 #!/bin/bash 
2 sj ale Sew cus tor illustrative jomooses Gc 
3 # Run it at your own peril -- it WILL freeze your system. 
4 
5 while true Endless loop. 
6 do 
7) $0 & This script invokes itself 
8 + forks an infinite number of times 
9 + until the system freezes up because all resources exhausted. 
10 done This is the notorious "sorcerer's appentice" scenario. 
idl 
12 exit 0 Will not exit here, because this script will never terminat 
A ulimit -Hu XX (where XX is the user process limit) in /etc/profile would abort this script 
when it exceeded the preset limit. 
quota 
Display user or group disk quotas. 
setquota 
Set user or group disk quotas from the command-line. 
umask 


User file creation permissions mask. Limit the default file attributes for a particular user. All files 
created by that user take on the attributes specified by umask. The (octal) value passed to umask 
defines the file permissions disabled. For example, umask 022 ensures that new files will have at 
most 755 permissions (777 NAND 022). [8] Of course, the user may later change the attributes of 
particular files with chmod. The usual practice is to set the value of umask in /etc/profile 
and/or -/.bash profile (see Appendix G). 


Example 17-10. Using umask to hide an output file from prying eyes 


#!/bin/bash 
# rotl3a.sh: Same 


# Usage: ./roti3a. 
# or a f Teton AL Sha) 
# or of iege L BAL. 


umask 177 


OUTFILE=decrypted. 


= 4 


as “irocls.cin" exiget, Isle WeLreS oviro To "secus" ilg. 


sh filename 
sh «filename 
sh and supply keyboard input (stdin) 


# File creation mask. 
Files created by this script 
#+ will have 600 permissions. 


+ 


EXE # Results output to file "decrypted.txt" 
#+ which can only be read/written 
# by invoker of script (or root). 


=] ey) (ap de» (3 Iss) J) (C Wey (9s! cp teny (On) dex (8) ISS) d 


cat "S@" | tr cUa-zA-Z' 'n-za-mN-ZA-M!' > SOUTFILE 
# ^^ omg trom stein gue m fle. MSS SSMS Our purre direct edir onale 


18 


19 exit 0 
rdev 
Get info about or make changes to root device, swap space, or video mode. The functionality of rdev 
has generally been taken over by lilo, but rdev remains useful for setting up a ram disk. This is a 
dangerous command, if misused. 
Modules 
Ismod 
List installed kernel modules. 
bash$ lsmod 
Module Size Used by 
autofs 9456 2 (autoclean) 
opl3 43.3 0 0 
serial cs 5456 0 (unused) 
sb 34752 0 
uart401 6384 0 [sb] 
sound 58368 0 [opl3 sb uart401] 
soundlow 464 0 [sound] 
soundcore 2800 6 [sb sound] 
ds 6448 2 leesexmi csl 
182365 22928 2 
pcmcia core 45984 0 [serial cs ds i82365] 
2^) Doing a cat /proc/modules gives the same information. 
insmod 
Force installation of a kernel module (use modprobe instead, when possible). Must be invoked as 
root. 
rmmod 
Force unloading of a kernel module. Must be invoked as root. 
modprobe 
Module loader that is normally invoked automatically in a startup script. Must be invoked as root. 
depmod 
Creates module dependency file. Usually invoked from a startup script. 
modinfo 
Output information about a loadable module. 
bash$ modinfo hid 
filename: /lib/modules/2.4.20-6/kernel/drivers/usb/hid.o 
description: "USB HID support drivers" 
author: "Andreas Gal, Vojtech Pavlik <vojtech@suse.cz>" 
license: MGE 
Miscellaneous 
env 


Runs a program or script with certain environmental variables set or changed (without changing the 
overall system environment). The [varname-xxx] permits changing the environmental variable 
varname for the duration of the script. With no options specified, this command lists all the 


environmental variable settings. [9] 


$^; The first line of a script (the "sha-bang" line) may use env when the path to the shell or 


interpreter is unknown. 


Idd 


watch 


strip 


rdist 


1 #! /usr/bin/env perl 
2 
3 prime “Weis Peril sCrior wall mwa, Va"p 
4 print "even when I don't know where to find Perl.\n"; 
5 
6 Coce row jOMEIOILS CHROSS-OLCIETOMM Sees, 
7 where the Perl binaries may not be in th xpected plac 
8 Thanks, S.C. 
Or even ... 
1 #!/bin/env bash 
2 Queries the SPATH enviromental variable for the location of bash. 
3 Therefore 
4 This script will run where Bash is not in its usual place, in /bin. 
5 


Show shared lib dependencies for an executable file. 


bash$ ldd /bin/ls 
Lise .80,6 => /115/1i1leg.segG (401000101) 
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) 


Run a command repeatedly, at specified time intervals. 
The default is two-second intervals, but this may be changed with the -n option. 


1 watch -n 5 tail /var/log/messages 
2 # Shows tail end of system log, /var/log/messages, every five seconds. 


$^) Unfortunately, piping the output of watch command to grep does not work. 


Remove the debugging symbolic references from an executable binary. This decreases its size, but 
makes debugging it impossible. 


This command often occurs in a Makefile, but rarely in a shell script. 
List symbols in an unstripped compiled binary. 


Remote distribution client: synchronizes, clones, or backs up a file system on a remote server. 


17.1. Analyzing a System Script 


Using our knowledge of administrative commands, let us examine a system script. One of the shortest and 
simplest to understand scripts is "killall," [10] used to suspend running processes at system shutdown. 


Example 17-11. killall, from /etc/rc.d/init.d 


!/bin/sh 
--» Comments added by the author of this document marked by "# --»". 


==> lis 1S pare oit tle "re" eoriptE package 
--» by Miquel van Smoorenburg, <miquels@drinkel.nl.mugnet.org>. 


==> Iais particular seript scene Co be Weel het / FC SjoScuicne 
--» (may not be present in other distributions). 


Bring down all unneeded services that are still running 
* (there shouldn't be any, so this is just a sanity check) 


for i in /var/lock/subsys/*; do 


NPRPPPP PPP PY 
O (o 0» 1 O Oi i (). M. IB. O xo 0 -1 O OI i QI 


--» Standard for/in loop, but since "do" is on same line, 
==> iit 3 necessary i9 gel Vp. 
Check if the script is there. 
! -f Si | && continue 
--» This is a clever use of an "and list", equivalent to: 
==> ni [ | es VS" Jp chan Col ime 
AL 
22 Get the subsystem name. 
23 subsys=$ {i#/var/lock/subsys/} 
24 --> Match variable name, which, in this case, is the file name. 
25 ==> This 3e ila xact equivalent of subsys= basename $i'. 
26 
2 --» It gets it from the lock file name 
28 c4 (if there is e lock tulle, 
29 --»-* that's proof the process has been running). 
30 --» See the "lockfile" entry, above. 
Sil 
32 
919 Bring the subsystem down. 
34 ai | c /etc/:ie.eu isi. SsulosysamuE |p then 
35 d Che 2S o C imie ool SSO VS oie SECO 
35 else 
37 PREC) BE (UJ dione .C/Ssussys SEOD 
38 # --» Suspend running jobs and daemons. 
3$ # --» Note that "stop" is a positional parameter, 
40 i^ B mou dm shell Igei ie ais. 
41 iE at 
42 done 


That wasn't so bad. Aside from a little fancy footwork with variable matching, there is no new material there. 


Exercise 1. In /etc/rc.d/init.d, analyze the halt script. It is a bit longer than killall, but similar in 
concept. Make a copy of this script somewhere in your home directory and experiment with it (do nof run it as 
root). Do a simulated run with the -vn flags (sh -vn scriptname). Add extensive comments. Change 
the "action" commands to "echos". 


Exercise 2. Look at some of the more complex scripts in /etc/rc.d/init.d.Seeif you can understand 


parts of them. Follow the above procedure to analyze them. For some additional insight, you might also 
examine the file sysvinitfilesin /usr/share/doc/initscripts-?.??, which is part of the 
"initscripts" documentation. 


Notes 


[1] Thisis the case on a Linux machine or a UNIX system with disk quotas. 
[2] The userdel command will fail if the particular user being deleted is still logged on. 
3 


[3] For more detail on burning CDRs, see Alex Withers' article, Creating CDs, in the October, 1999 issue 
of Linux Journal. 


[4] The -c option to mke2fs also invokes a check for bad blocks. 


[5] Since only root has write permission in the /var/ lock directory, a user script cannot set a lock file 
there. 


[6] Operators of single-user Linux systems generally prefer something simpler for backups, such as tar. 


[7] As ofthe version 4 update of Bash, the -f and -c options take a block size of 512 when in POSIX 
mode. Additionally, there are two new options: —b for socket buffer size, and -T for the limit on the 
number of threads. 


NAND is the logical not-and operator. Its effect is somewhat similar to subtraction. 


[8] 
[9] In Bash and other Bourne shell derivatives, it is possible to set variables in a single command's 
environment. 


1 varl-valuel var2-value2 commandXXX 
2 4 Svarl and $var2 set in the environment of 'commandXXX' only. 


[10] The killall system script should not be confused with the killall command in /usr/bin. 
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Part 5. Advanced Topics 


At this point, we are ready to delve into certain of the difficult and unusual aspects of scripting. Along the 
way, we will attempt to "push the envelope" in various ways and examine boundary conditions (what happens 


when we move into uncharted territory?). 
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Chapter 18. Regular Expressions 


... the intellectual activity associated with 
software development is largely one of gaining 
insight. 


--Stowe Boyd 
To fully utilize the power of shell scripting, you need to master Regular Expressions. Certain commands and 


utilities commonly used in scripts, such as grep, expr, sed and awk, interpret and use REs. As of version 3, 
Bash has acquired its own RE-match operator: =~. 


18.1. A Brief Introduction to Regular Expressions 


An expression is a string of characters. Those characters having an interpretation above and beyond their 
literal meaning are called metacharacters. A quote symbol, for example, may denote speech by a person, 
ditto, or a meta-meaning [1] for the symbols that follow. Regular Expressions are sets of characters and/or 
metacharacters that match (or specify) patterns. 


A Regular Expression contains one or more of the following: 


* A character set. These are the characters retaining their literal meaning. The simplest type of Regular 
Expression consists only of a character set, with no metacharacters. 

e. 
An anchor. These designate (anchor) the position in the line of text that the RE is to match. For 


example, ^, and $ are anchors. 
* Modifiers. These expand or narrow (modify) the range of text the RE is to match. Modifiers include 
the asterisk, brackets, and the backslash. 


The main uses for Regular Expressions (REs) are text searches and string manipulation. An RE matches a 
single character or a set of characters -- a string or a part of a string. 


* The asterisk -- * -- matches any number of repeats of the character string or RE preceding it, 
including zero instances. 


"1133*" matches 11 + one or more 3's: 113, 1133, 1133333, and so forth. 
The dot -- . -- matches any one character, except a newline. [2] 


"13." matches 13 + at least one of any character (including a space): 
1133, 11333, but not 13 (additional character missing). 


See Example 16-18 for a demonstration of dot single-character matching. 
The caret -- ^ -- matches the beginning of a line, but sometimes, depending on context, negates the 
meaning of a set of characters in an RE. 


The dollar sign -- $ -- at the end of an RE matches the end of a line. 

"XXX$" matches XXX at the end of a line. 

"^$" matches blank lines. 

Brackets -- [...] -- enclose a set of characters to match in a single RE. 

"[xyz]" matches any one of the characters x, y, or z. 

"[c-n]" matches any one of the characters in the range c to n. 

"[B-Pk-y]" matches any one of the characters in the ranges B to P and k to y. 
"[a-z0-9]" matches any single lowercase letter or any digit. 


"[^b-d]" matches any character except those in the range b to d. This is an instance of ^ negating or 
inverting the meaning of the following RE (taking on a role similar to ! in a different context). 


Combined sequences of bracketed characters match common word patterns. "[Yy][Ee][Ss]" matches 
yes, Yes, YES, yEs, and so forth. "[0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]" matches any 
Social Security number. 

e. 
The backslash -- \ -- escapes a special character, which means that character gets interpreted literally 
(and is therefore no longer special). 


A "\$" reverts back to its literal meaning of "$", rather than its RE meaning of end-of-line. Likewise a 
"^W" has the literal meaning of "V". 
e. 


Escaped "angle brackets" -- \<...\> -- mark word boundaries. 
The angle brackets must be escaped, since otherwise they have only their literal character meaning. 
"\<the\>" matches the word "the," but not the words "them," "there," "other," etc. 


bash$ cat textfile 
This is line 1, of which there is only one instance. 
This is the only instance of line 2. 
dme as line 3, euo done 
This is line 4. 


bash$ grep 'the' textfile 

This is line 1, of which there is only one instance. 
This is the only instance of line 2. 

Mase le lite S5 reweYowElexewe iNe. 


bash$ grep '\<the\>' textfile 
This is the only instance of line 2. 


The only way to be certain that a particular RE works is to test it. 


WWIII? je Sie a LS 


grep Yililsiss this file. 


his line contains the number 113. 

his line contains the number 13. 

his line contains the number 133. 

his line contains the number 1133. 

has lane Contains the number 1133127 
his line contains the number 1112. 

Ins line comceanns icles manosi Wil SSeS 12 - 
LS Tihs lame contains no numberms ewt all, 


bash$ grep "1133*" tstfile 
Run greo ViilsseY om ills Lle, 
This line contains the number 113. 
This line contains the number 1133. 
I seameseoniüsadmsashcemumli cce esi 2c 
Tass dling Gomeasing rre jubere  1L1 9153302: 112 ., 


i 
2 
3 
4 
S 
6 
7 
8 
9 
10 
ial 
4.2 


HHHHHHHH 


e Extended REs. Additional metacharacters added to the basic set. Used in egrep, awk, and Perl. 


* The question mark -- ? -- matches zero or one of the previous RE. It is generally used for matching 
single characters. 

e. 
The plus -- + -- matches one or more of the previous RE. It serves a role similar to the *, but does not 
match zero occurrences. 


# GNU versions of sed and awk can use "+", 
# but it needs to be escaped. 


d 

2 

E 

4 echo alilia | sed -ne '/al\+b/p' 
5 eno allla || greo Valya’ 

6 echo alllb | gawk '/al+b/' 

7 # All of above ar quivalent. 
8 
9 


ur Uineinkes, SoC. 
e Escaped "curly brackets" -- \{ \} -- indicate the number of occurrences of a preceding RE to match. 


It is necessary to escape the curly brackets since they have only their literal character meaning 
otherwise. This usage is technically not part of the basic RE set. 


"[0-0N 5X)" matches exactly five digits (characters in the range of 0 to 9). 


$^; Curly brackets are not available as an RE in the "classic" (non-POSIX compliant) 
version of awk. However, the GNU extended version of awk, gawk, has the 
--re-interval option that permits them (without being escaped). 


bash$ echo 2222 | gawk --re-interval '/2(3)/' 
2222 


Perl and some egrep versions do not require escaping the curly brackets. 
e. 


Parentheses -- ( ) -- enclose a group of REs. They are useful with the following 


substring extraction using expr. 
e The -- | -- "or" RE operator matches any of a set of alternate characters. 


operator and in 


bash$ egrep 're(ale)d' misc.txt 
People who read seem to be better informed than those who do not. 
The clarinet produces sound by the vibration of its reed. 


Some versions of sed, ed, and ex support escaped versions of the extended Regular Expressions 
described above, as do the GNU utilities. 


e POSIX Character Classes. [:class:] 


This is an alternate method of specifying a range of characters to match. 

: alnum:] matches alphabetic or numeric characters. This is equivalent to ACZa-z0-9. 

:alpha:] matches alphabetic characters. This is equivalent to A-Za-z. 

:blank:] matches a space or a tab. 

:entrl:] matches control characters. 

:digit:] matches (decimal) digits. This is equivalent to 0—9. 

[:graph:] (graphic printable characters). Matches characters in the range of ASCII 33 - 126. This 

is the same as [:print:], below, but excluding the space character. 

e [: lower: ] matches lowercase alphabetic characters. This is equivalent to a-z. 

e [:print:] (printable characters). Matches characters in the range of ASCII 32 - 126. This is the 
same as [: graph: ], above, but adding the space character. 


Ce Oe ee 03 


e [:space: ] matches whitespace characters (space and horizontal tab). 
e [: upper: ] matches uppercase alphabetic characters. This is equivalent to A-Z. 
e [:xdigit:] matches hexadecimal digits. This is equivalent to 0-9A-Fa-f. 


(D POSIX character classes generally require quoting or double brackets ([[ ]]). 


bash$ grep [[:digit:]] test.file 


abc=723 
eG xm 
2, ase [I Sxessow; == | | eckieates |) Jl # Numerical input? 
3 then # POSIX char class 
4 if [I $acol =~ [[:alpha:]] ]] # Number followed by a letter? Illegal! 
5 d 
6 # From ktour.sh example script. 


These character classes may even be used with globbing, to a limited extent. 


bash$ ls -1 ?[[:digit:]][[:digit:]]? 
STASI iL loose loeo 


0 Aug 21 14:47 a33b 


POSIX character classes are used in Example 16-21 and Example 16-22. 


Sed, awk, and Perl, used as filters in scripts, take REs as arguments when "sifting" or transforming files or I/O 
streams. See Example A-12 and Example A-16 for illustrations of this. 


The standard reference on this complex topic is Friedl's Mastering Regular Expressions. Sed & Awk, by 
Dougherty and Robbins, also gives a very lucid treatment of REs. See the Bibliography for more information 
on these books. 


Notes 


[1] A meta-meaning is the meaning of a term or expression on a higher level of abstraction. For example, 
the literal meaning of regular expression is an ordinary expression that conforms to accepted usage. 
The meta-meaning is drastically different, as discussed at length in this chapter. 


[2] Since sed, awk, and grep process single lines, there will usually not be a newline to match. In those 
cases where there is a newline in a multiple line expression, the dot will match the newline. 
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19 
20 


!/bin/bash 


sacl e ike oe / Telly! «<< 
linel 

line2 

EOF 

OUTPUT; 

[linel 

line2] 


echo 


awk "4 SWSSi UN S252 sr 
line 1 

imc? 

EOF 

# OUTPUT: 

# line 


(/line.1/) 


EOF # Here Document 


(jor sine} PY << 


EOF 


ZA i "bsveuadeeho oo 
25 
26 exit 0 
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18.2. Globbing 


Bash itself cannot recognize Regular Expressions. Inside scripts, it is commands and utilities -- such as sed 
and awk -- that interpret RE's. 


Bash does carry out filename expansion [1] -- a process known as globbing -- but this does not use the 
standard RE set. Instead, globbing recognizes and expands wild cards. Globbing interprets the standard wild 
card characters [2] -- * and ?, character lists in square brackets, and certain other special characters (such as ^ 
for negating the sense of a match). There are important limitations on wild card characters in globbing, 
however. Strings containing * will not match filenames that start with a dot, as, for example, .bashrc. [3] 
Likewise, the ? has a different meaning in globbing than as part of an RE. 


bash$ Is -I 
cotel 2 
—rW—rw-rf-- 
—EW-—rW-f-- 
—rw—rw-r-- 
—EW-EW-f--— 
—rW—fw-f-- 


bash$ 1s -1 
-rw-rw-r-- 


bash$ 1s -1 
-rw-rw-r-- 
-rw-rw-r-- 


bash$ 1s -1 
-rw-rw-r-- 
-rw-rw-r-- 
-rw-rw-r-- 


bash$ 1s -1 
-rw-rw-r-- 
-rw-rw-r-- 
-rw-rw-r-- 


bash$ 1s -1 
-rw-rw-r-- 
-rw-rw-r-- 
-rw-rw-r-- 


bozo 
bozo 
bozo 
bozo 
bozo 


PREP PP 


t?.sh 
iL bozo 


[ab]* 
iL lex 
iL oza 


[a-c]* 
iL lee) 
iL ToXoXAe) 
iL exo) 


[^ab]* 
iL oza 
iL loxeyzAco) 
iL bozo 


bozo 
bozo 
bozo 
bozo 
bozo 


bozo 


bozo 
bozo 


bozo 
bozo 
bozo 


bozo 
bozo 
bozo 


(b*,c*,*est*) 


iL lexee) 
IN bozo 
IDOA 


bozo 
bozo 
bozo 


466 


TSR 


Aug 6 
Aug 6 
Aug 6 
Aug 6 
Jul S9 
Aug 6 
Aug 6 
Aug 6 
Aug 6 
Aug 6 
Aug 6 
Aug 6 
Aug 6 
Jeu si) 
Aug 6 
Aug 6 
Jul 30 


ili} 8 
iL 8 
iioi: 
eS 
(DIOS 


ie 


ALIS B 
ILI S 


JL e 
Leg 
Wok 


LEE 
ys 
ODE 


LEE 
Ileje 
09E 


42 
42 
42 
48 
02 


48 


42 
42 


42 
42 
42 


42 
48 
02 


42 
42 
02 


(QV ter w 
PPR 


152 c slm 
testl.txt 


i2) c ilm 


a 
A 


Cra 
15:24 Sila 
testi CXE 


Dol 
(indi 
testi o ESTE, 


Bash performs filename expansion on unquoted command-line arguments. The echo command demonstrates 


this. 


bash$ echo * 


Bind deo dL. sdb A edm. eese s Tode 


bash$ echo t* 


t2.sh testi. 


EXE 


bash$ echo t?.sh 


1622 c (elm 


$^) It is possible to modify the way Bash interprets special characters in globbing. A set -f command 
disables globbing, and the nocaseglob and nu11glob options to shopt change globbing behavior. 


See also Example 11-4. 


Notes 


[1] Filename expansion means expanding filename patterns or templates containing special characters. For 
example, example.??? might expand to example.001 and/or example.txt. 


[2] A wild card character, analogous to a wild card in poker, can represent (almost) any other character. 


[3] Filename expansion can match dotfiles, but only if the pattern explicitly includes the dot as a literal 


character. 
i y pa lbashre Will not expand to -/.bashrc 
2 ~/?bashre Neither will this. 
S Wild cards and metacharacters will NOT 
4 + expand to a dot in globbing. 
5 
6 «s Poll asinre Will expand to -/.bashrc 
T cf bazire Likewise. 
8 ~/.bashr* Likewise. 
E 


10 s» Serting the YelotgileloY gor om iius Elis (Quit. 


JA. e» SMloveamkci, SiG. 
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Chapter 19. Here Documents 
Here and now, boys. 
--Aldous Huxley, Island 


A here document is a special-purpose code block. It uses a form of I/O redirection to feed a command list to 
an interactive program or a command, such as ftp, cat, or the ex text editor. 


1 COMMAND ««InputComesFromHERE 
E 
3 InputComesFromHERE 


A limit string delineates (frames) the command list. The special symbol «« designates the limit string. This 
has the effect of redirecting the output of a file into the st din of the program or command. It is similar to 
interactive-program < command-file, where comnand- file contains 


1 command #1 
2 command #2 
3 


The here document alternative looks like this: 


#!/bin/bash 

interactive-program <<LimitString 
command #1 

command #2 


(nl gm 9 fe» [25 


6 LimitString 
Choose a limit string sufficiently unusual that it will not occur anywhere in the command list and confuse 
matters. 


Note that here documents may sometimes be used to good effect with non-interactive utilities and commands, 
such as, for example, wall. 


Example 19-1. broadcast: Sends message to everyone logged in 


1 #!/bin/bash 

2 

3 wall ««zzz23EndOfMessagezzz23 

4 E-mail your noontime orders for pizza to the system administrator. 
3 (Add an extra dollar for anchovy or mushroom topping.) 
6 Additional message text goes her 

T Note: 'wall' prints comment lines. 

8 zzz23EndOfMessagezzz23 

9 
10 Could have been done mor fficiently by 
d wall «message-file 
12 However, embedding the message template in a script 
i3 pr is e GqUlck-anmc-—cairty gue-oii Solne lons, 
14 


15 exit 


Even such unlikely candidates as the vi text editor lend themselves to here documents. 


Example 19-2. dummyfile: Creates a 2-line dummy file 


!/bin/bash 


Nonimteractiive WISE toe Vaya’) ee edie ~, seal Ile, 
Emulates 'sed'. 


E BADARGS-85 


ste. [pov WeLw 

then 
echo "Usage: ~basename $0! filename" 
exit $E BADARGS 

3E dL 


TARGETFILE-$1 


# Insert 2 lines in file, then save. 

# - Begin here document - # 
vi STARGETFILE <<x23LimitStringx23 

i 

Haks ays Lines db (exe Ca xample fil 


NPRPRPPRP PPP PY 
O (o 0» -1 OY Oi i$ (). NM PS. O xo 0 -1 O O1 iS CQ Io E 


Zit "sce ales lleve 2 Oi iem xample fil 

22 IS | 

2:987 7 

24 x23LimitStringx23 

25 = End here document d 
26 

27 4 Note that ^[ above is a literal escape 


28 #+ typed by Control-V «Esc». 

29 

30 # Bram Moolenaar points out that this may not work with 'vim' 
31 #+ because of possible problems with terminal interaction. 

82 


33 exit 


The above script could just as effectively have been implemented with ex, rather than vi. Here documents 
containing a list of ex commands are common enough to form their own category, known as ex scripts. 


1 #!/bin/bash 

2 # Replace all instances of "Smith" with "Jones" 
SL sper aber Seales qyalicia @ “exe drablheusvewe: Wie lox. 
4 

5 ORIGINAL-Smith 

6 REPLACEMENT=Jones 

7 

8 for word in $(fgrep -L SORIGINAL *.txt) 

3 clo 

10 # 

Til ex $word <<EOF 

12 :$s/SORIGINAL/SREPLACEMENT/g 

ILS :Wq 

14 EOF 

LS # :$s is the "ex" substitution command. 
16 # :wq is write-and-quit. 

Ly # 

18 done 


Analogous to "ex scripts" are cat scripts. 


Example 19-3. Multi-line message using cat 


#!/bin/bash 


# ‘echo' is fine for printing single line messages, 
#+ but somewhat problematic for for message blocks. 
# A 'cat' here document overcomes this limitation. 


cat ««End-of-message 


This is line 1 of the message. 
This is line 2 of the message. 
This is line 3 of the message. 
This is line 4 of the message. 
This is the last line of the messag 


End-of-message 


Replacing line 7, above, with 
dE cat > SNewfile ««End-of-messag 


A^A^A^A^A^AAA^AA 


+ writes the output to the file $Newfile, rather than to stdout. 


Mop pop pop PPP PY 
O (o 0» -1 O Oi iS (). N I2. O xo 0 -1 O O1 iS CQ I 


N N 
N he 


Essie (0) 


N N 
JM W 


N 
[91] 


# 
# Code below disabled, due to "exit 0" above. 


ISS) ISS} 
1D 


N 
[99] 


# S.C. points out that the following also works. 
emo "" 

This is line 1 of the message. 

This is line 2 of the message. 

This is line 3 of the message. 

This is line 4 of the message. 

This is the last line of the messag 


CO CO CO CO CO CO NO 
(On. dex. (63) IN) f S&S) We) 


36 4 However, text may not include double quotes unless they ar scaped. 


The - option to mark a here document limit string (<<-LimitString) suppresses leading tabs (but not 
spaces) in the output. This may be useful in making a script more readable. 


Example 19-4. Multi-line message, with tabs suppressed 


#!/bin/bash 
# Same as previous example, but... 


# The - option to a here document << 
#+ Suppresses leading tabs in the body of the document, 
#+ but *not* spaces. 


cat ««-ENDOFMESSAGE 
This is line 1 of the message. 

This is line 2 of the message. 

This is line 3 of the message. 

This is line 4 of the message. 

This is the last line of the messag 

ENDOFMESSAGE 

The output of the script will be flush left. 

Leading tab in each line will not show. 


Above 5 lines of "message" prefaced by a tab, not spaces. 
Spaces) noL aErecued by <<- 


©) Wer (es. «| (ex, (On dex. GIS) [SS ke (es SI] @) (a des E S du» (3 


(S 


21 # Note that this option has no effect on *embedded* tabs. 
22 
23 exit 0 


A here document supports parameter and command substitution. It is therefore possible to pass different 
parameters to the body of the here document, changing its output accordingly. 


Example 19-5. Here document with replaceable parameters 


1 #!/bin/bash 

2 4 Another 'cat' here document, using parameter substitution. 
E 

4 4$ Try it with no command-line parameters, ./scriptname 

5 # Try it with one command-line parameter, ./scriptname Mortimer 
6 # Try it with one two-word quoted command-line parameter, 

7 d ./scriptname "Mortimer Jones" 

8 

9 CMDLINEPARAM=1 # Expect at least command-line parameter. 
10 
11 if [ $4 -ge SCMDLINEPARAM ] 
12 then 
d NAME-$1 # If more than one command-line param, 
14 #+ then just take the first. 
15 else 
16 NAME="John Doe" # Default, if no command-line parameter. 
dL) dea 

18 

19 RESPONDENT-"the author of this fine script" 
20 
2 
22 cat ««Endofmessage 
23 
24 Hello, there, $NAME. 
25 Greetings to you, $NAME, from SRESPONDENT. 
26 
2,1] This comment shows up in the output (why?). 
28 
29 Endofmessage 
30 
Sil Note that the blank lines show up in the output. 
32 So does the comment. 
33 
34 exit 


This is a useful script containing a here document with parameter substitution. 


Example 19-6. Upload a file pair to Sunsite incoming directory 


!/bin/bash 
upload.sh 


Upload file pair (Filename.lsm, Filename.tar.gz) 
* to incoming directory at Sunsite/UNC (ibiblio.org). 
Filename.tar.gz is the tarball itself. 
Filename.lsm is the descriptor file. 
Sunsite requires "lsm" file, otherwise will bounce contributions. 


or o9) «p (xj Gal de» (e» [R9 [5 


E ARGERROR-85 


if [ -z "$q" ] 


K} H H H H H H H Eg 
(25) We) (ey =] fp) (Ont des (ed) fe f SS 
[0] 

a 
[Sr 
o 


"Usage: ~basename $0" 
exit $E ARGERROR 


$3 


Filename-'basenam 

2a 

22 Server-"ibiblio.org" 

23 Directory-"/incoming/Linux" 
24 

25 

26 

27 


N 
[99] 


binary 
bell 
cd SsDirectory 


CO CO CO CO CO CO CO CO CO CO Dd 
We) (ex cp (exp Gy des (3) ho [3 9» ve 


bye 
40 End-Of-Session 
ar 

42 exe 0 


Password="your.e-mail.address" 


iro in Serrar <Aadinvel—Oie—SSs) sii 
# -n option disables auto-logon 


user anonymous "SPassword" 


put "SFilename.lsm" 
put "SFilename.tar.gz" 


Filename-to-upload" 


# Strips pathname out of file name. 


# These need not be hard-coded into script, 
#+ but may instead be changed to command-line argument. 


# Change above to suit. 


# If 


this doesn't work, then try: 


# quote user anonymous "SPassword" 


# Ring 'bell' after each file transfer. 


Quoting or escaping the "limit string" at the head of a here document disables parameter substitution within its 
body. The reason for this is that quoting/escaping the limit string effectively escapes the $, `, and V special 
characters, and causes them to be interpreted literally. (Thank you, Allen Halsey, for pointing this out.) 


Example 19-7. Parameter substitution turned off 


#!/bin/bash 


NAME="John Doe" 


Greetings to you, 


Endofmessage 


(= 5 We) Cer — (ex, (ab dE GS [eo [— c5» wer Csr c ep rl gES (93. de» (3 


WN 


# A 'cat' here-document, 


cat <<'Endofmessage' 


Hello, there, SNAME. 


SNAME, from SR 


but with parameter substitution disabled. 


RESPONDENT="the author of this fine script" 


ESPOND 


ENT. 


No parameter substitution when the "limit string" is quoted or escaped. 
Either of the following at the head of the here document would have 
+ the same effect. 
cat ««"Endofmessage" 
cat <<\Endofmessage 


22 t And, likewise: 

24 cat ««"SpecialCharTest" 

26 Directory listing would follow 

27 if limit string were not quoted. 

25 "A5 I 

30 Arithmetic expansion would take place 
31 if limit string were not quoted. 

32 &((B5 s 3) 

34 A a single backslash would echo 

35 if limit string were not quoted. 


86 y 


38 SpecialCharTest 


41 exit 


Disabling parameter substitution permits outputting literal text. Generating scripts or even program code is 
one use for this. 


Example 19-8. A script that generates another script 


#!/bin/bash 
# generate-script.sh 
# Based on an idea by Albert Reiner. 


OUTFILE=generated.sh # Name of the file to generat 


"Here document containing the body of the generated script. 
( 
Ceu «e UIS V 
! /bin/bash 


echo "This is a generated shell script." 
Note that since we are inside a subshell, 
+ we can't access variables in the "outside" script. 


cho "Generated file will be named: S$OUTFILE" 
Above line will not work as normally expected 
+ because parameter expansion has been disabled. 


NPRPRPPRP PPP PY 
O (o 0» -1 O Oi i5 (). NM. I. O xo 0 -1 O O1 4 C I ES 


Zak Instead, the result is literal output. 

22 

23 a=7 

24 b=3 

25 

26 let "g = Se S" 

27 Cine "e = Sov 

28 

29) exei @ 

30 EOF 

Si) > SOUE 

32 

33 

34 Quoting the 'limit string' prevents variable expansion 
35 #+ within the body of the above "here document.' 

36 This permits outputting literal strings in the output file. 


37 

398 ae | = VOXOUMUEIILE 
39 then 

40 chmod 755 SOUTFIL!I 
41 # Make the genera 
42 else 

43 echo "Problem in creating file: \"SOUTFILE\"" 

44 fi 

45 

46 # This method can also be used for generating 

47 #+ C programs, Perl programs, Python programs, Makefiles, 
48 #+ and the like. 

49 

50 exit 0 


cat ew 


ql sé ad) xecutabl 


It is possible to set a variable from the output of a here document. This is actually a devious form of command 
substitution. 


variable=S$ (cat ««SETVAR 
This variable 

runs over multiple lines. 
SETVAR) 


ep) (Gal PSS (5) a 


echo "Svariable" 


A here document can supply input to a function in the same script. 


Example 19-9. Here documents and functions 


#!/bin/bash 
# here-function.sh 


GetPersonalData () 
{ 
read firstname 
read lastname 
read address 
read city 
read state 
read zipcode 
} # This certainly looks like an interactive function, but... 


# Supply input to the above function. 
GetPersonalData <<RECORDOO1 


NPRPRPP PPP PP 
O (o 00 -1 O O1 i$ (0. NM. IB. O xo 0 -1 O O1 i8 CQ) I ES 


Bozo 
Bozeman 
2726 Nondescript Dr. 
Baltimore 
21 MD 
22 2122 
23 RECORD OOM 
24 
25 
26 echo 
27 echo "Sfirstname $1astname" 
28 echo WSievolclieesis! 
29 echo "Scity, $state $zipcode" 
30 echo 
ES 
32 exit 0 


It is possible to use : as a dummy command accepting output from a here document. This, in effect, creates an 


"anonymous" here document. 


Example 19-10. "Anonymous" Here Document 


#!/bin/bash 


<<TESTVARIABLES 


TESTVARIABLES 


L 

2 

oe 

4 S{HOSTNAME?}${USER?}S{MAIL?} # Print error message if one of th 
5) 

6 

»: 


exe $9? 


variables not set. 


į ) A variation of the above technique permits "commenting out" blocks of code. 


Example 19-11. Commenting out a block of code 


1 #!/bin/bash 
2 # commentblock.sh 


<<COMMENTBLOCK 
echo "This line will not echo." 
This is a comment line missing the "#" prefix. 
This is another comment line missing the "#" prefix. 


&*Ql!ltr- 

The above line will cause no error message, 
because the Bash interpreter will ignore it. 
COMMENTBLOCK 


echo "Exit value of above \"COMMENTBLOCK\" is $?." # 0 
# No error shown. 
STNG 


(cob «| oy, (nb des 403) IS) J) Sr We) se) sa) oy, Gut des (o3 


Abg, The above technique also comes in useful for commenting out 
20 #+ a block of working code for debugging purposes. 

Zl This saves having to put a "4" at the beginning of each line, 
22 #+ then having to go back and delete each "4$" later. 

29 Note that the use of of colon, above, is optional. 


25 echo "Just before commented-out code block." 


26 The lines of code between the double-dashed lines will not execute. 


28 : ««DEBUGXXX 
219) ieee Galle ang 7 
30 do 

Lo ecat vo minke 
32 done 

33 DEBUGXXX 


34 # 


35 echo "Just after commented-out code block." 


Sy exit 0 


41 HEHEHE E E E E E E E E EEE E ERE EEE E EEE HERE EEE EEE GREE EEE HEE HEE HEHE HERE 
42 Note, however, that if a bracketed variable is contained within 
43 #+ the commented-out code block, 

44 #+ then this could cause problems. 

45 for example: 

46 

47 

48 #/!/bin/bash 

49 

50) : ««COMMENTBLOCK 

xil exeimo "imis Iau: waLILlb more (exeo. 

52 &* R! !++= 

53 Siroo bar bazz?) 

54 $(rm -rf /tmp/foobar/) 

55 $(touch my build directory/cups/Makefile) 

56 COMMENTBLOCK 


59 $ sh commented-bad.sh 
60 commented-bad.sh: line 3: foo bar bazz: parameter null or not set 


62 4$ The remedy for this is to strong-quote the 'COMMENTBLOCK' in line 49, above. 


64 : ««'COMMENTBLOCK' 


66 4 Thank you, Kurt Pfeifle, for pointing this out. 


* 


į ) Yet another twist of this nifty trick makes "self-documenting" scripts possible. 


Example 19-12. A self-documenting script 


#!/bin/bash 
# self-document.sh: self-documenting script 
a? MioeliELesteiem Gif "eoim. sin 


DOC_REQUEST=70 


if [ "$1" = "-h" -o "$1" = "—-help" ] # Request help. 
then 
echo; echo "Usage: $0 [directory-name]"; echo 
sed --silent '/DOCUMENTATIONXX$/,/^DOCUMENTATIONXXS/p' "SO" | 
sed -e '/DOCUMENTATIONXX$/d'; exit $DOC REQUEST; fi 


<<DOCUMENTATIONXX 
List the statistics of a specified directory in tabular format. 


The command-line parameter gives the directory to be listed. 
If no directory specified or directory specified cannot be read, 
then list the current working directory. 


NPRPRPPRP PPP PY 
COMIDAGUHRWNHFOWOAIDGURWNE 


21 DOCUMENTATIONXX 

22 

Aor aw [p ew VSI So fo Se Wisi | 
24 then 

25 directory=. 

26 else 

27 directory-"$1" 

Zo. stab 


N 
Ko) 


30 echo Wiaisicsune rx "She ouo sU exo 

31 (printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAMENn" \ 
32 p Ile =1 "Scirectory” | see ide) | collumn =t 

33 

34 exit 0 


Using a cat script is an alternate way of accomplishing this. 


DOC_REQUEST=70 


i 

2 

Sj mie dp Wal  wejau xe. WSL c W elon | # Request help. 

4 then # Use a "cat script" 
5 cat <<DOCUMENTATIONXX 
6 
J 
8 


List the statistics of a specified directory in tabular format. 


The command-line parameter gives the directory to be listed. 
9 If no directory specified or directory specified cannot be read, 
10 then list the current working directory. 


12 DOCUMENTATIONXX 
13 exit $DOC REQUEST 
WA dea 


See also Example A-28, Example A-40, Example A-41, and Example A-42 for more examples of 
self-documenting scripts. 


$^; Here documents create temporary files, but these files are deleted after opening and are not accessible to 
any other process. 


bash$ bash -c 'lsof -a -p $$ -d0' << EOF 
» EOF 
lsof L23 lex Or REG 3,5) 0 30386 /tmp/t1213-0-sh (deleted) 


d Some utilities will not work inside a here document. 


The closing limit string, on the final line of a here document, must start in the first character position. 
There can be no leading whitespace. Trailing whitespace after the limit string likewise causes 
unexpected behavior. The whitespace prevents the limit string from being recognized. 


#!/bin/bash 


cho " " 


echo "This is line 1 of the message inside the here document." 
echo "This is line 2 of the message inside the here document." 
echo "This is the final line of the message inside the here document." 


il 
2 
3 
4 
5 em «SehemSE 3g 
6 
7 
8 


9 LimitString 

10 4^^^^Indented limit string. Error! This script will not behave as expected. 
1i 

12 echo " n 
13 


14 # These comments are outside the 'here document', 
15 #+ and should not echo. 


17 echo "Outside the here document." 


JG) esate © 


2l echo “iwesis Line hac loxewcicieus not 66ra. Y a gol love an Verw Cenimernsla 


D Some people very cleverly use a single ! as a limit string. But, that's not necessarily a good idea. 


1 # This works. 

A cat <<! 

3 Hello! 

4 ! Three more exclamations !!! 

5c 

6 

7 

( p JEXDHE 

9 ee <<} 
10 Hello! 

11 Single exclamation point follows! 
eZ! 
ig d 
14 # Crashes with an error message. 
15 
16 
17 4 However, the following will work. 
18 cat ««EOF 
19 Hello! 
20 Single exclamation point follows! 
2 
22 INO 


23 4 It's safer to use a multi-character limit string. 


For those tasks too complex for a here document, consider using the expect scripting language, which was 
specifically designed for feeding input into interactive programs. 


19.1. Here Strings 


A here string can be considered as a stripped-down form of a here document. 
It consists of nothing more than COMMAND ««« $WORD, 
where SWORD is expanded and fed to the stdin of COMMAND. 


As a simple example, consider this alternative to the echo-grep construction. 


1 # Instead of: 

Z a£ echo USWARV | grep =e SE s; ale Dp SVAR = cx I 
3 # etc. 

4 

S ap eye 

© zt grap =e; "sg" <<< VS WAR” 

7 then LÀ ed 

8 echo "SVAR contains the substring sequence \"txt\"" 
9) stab 
10 # Thank you, Sebastian Kaminski, for the suggestion. 


Or, in combination with read: 


SHELIA la a Seine; Oe wordas” 


reacli =r -a Words <<< UGG 
# The -a option to "read" 
#+ assigns the resulting values to successive members of an array. 


echo "esce word im Srii 1S 5 ${Words[0]}" This 

exeo “Secomel wore ium Strime lss ${Words[1]}" Le 

echo Vise! vacuo! aim Greine 1S8 ${Words[2]}" a 

echo Vinoumicin vuol am (Sursum; Les $(Words[3])" Steeg 
echo "habituel wore im Sremo LSE $(Words[4])" ui 

exeo “VSasxitin vorc! ain Sriime Leg ${Words[5]}" words. 
echo "Seventh word in String is: $(Words[6])" inu 

Past end of $String. 


|pobombpmopmwmn 
O40 NP O00 -10 0&0 N Dd 


16 # Thank you, Francisco Lobo, for the suggestion. 


Example 19-13. Prepending a line to a file 


!/bin/bash 
prepend.sh: Add text at beginning of file. 


Example contributed by Kenny Stauffer, 
+ and slightly modified by document author. 


E NOSUCHFILE-85 


read -p "File: " file # -p arg to 'read' displays prompt. 
aie | 2 =e UEExle" | 
then # Bail out if no such file. 


acne Wrile Grile nor Corme, Y 
exit $E NOSUCHFILE 
ipa 


|obombpmopmpmowmn 
OY O! 4 (0. I9. IP. O xo O0 -1 O OI i8 CQ) IN S 


each op "ples Ww qae 
cat - Sfile <<<Stitle > $file.new 


echo "Modified file is Sfile.new" 


exit # Ends script execution. 
from 'man bash': 
Here Strings 
A variant of here documents, the format is: 
«««word 
The word is expanded and supplied to the command on its standard input. 
Of course, the following also works: 


sed -e '1i\ 
audes 9 Sexe 


Example 19-14. Parsing a mailbox 


Mop p opp ppppDgÀ: 
O (o 00 1 O O1 i$ (). M P O «o 0 -1 O O1 4 CQ Io E 
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od 
=) 


!/bin/bash 
Script by Francisco Lobo, 

+ and slightly modified and commented by ABS Guide author. 
Used in ABS Guide with permission. (Thank you!) 


This script will not run under Bash versions < 3.0. 


E_MISSING_ARG=67 
if [ -z "$q" ] 


echo "Usage: $0 mailbox-file" 
exit $E_MISSING_ARG 


mbox_grep() # Parse mailbox file. 


{ 


declar i body=0 match=0 
declar a date sender 
declare mail header value 


while IFS= read -r mail 
# COINS Reset SIFS. 
# Otherwise "read" will strip leading & trailing space from its input. 


do 
aff [ Sia eve. Yeo V Jy] # Match "From" field in message. 
then 
(( loochy = 0 )») # "Zero out" variables. 
(( match = 0 )) 


unset date 


elif (( body )) 
then 
(( match )) 
# echo "Smail" 
# Uncomment above line if you want entire body of message to display. 


elre | Smari Jio them 


41 IFS-: read -r header value <<< "Smail" 
42 mw Misi SEr Ingi 
43 
44 case "Sheader" in 

qo iE] [ese] [Oo] jeu] 9 [LiL Svale =~ YS2" JT && ( matca JI pe 
46 Naten Eom S 

47 xe sal [Wel lks] ) reac -r -a data <<< U"S$welue" pp 

48 eoe 

49 Match "Date" line. 

50 Re] trel [Cel] lke] [Lacs d Tyv] [pie ]| acl] ) reac ie -a sender <<< USwedue" 2 
5 AUN 

52 Match IP Address (may be spoofed). 

53 esac 

54 

55 else 

56 (( body++ )) 

b (( match )) && 

58 echo "MESSAGE S{date:+of: $(date[*]) }" 

59 d Entire $date array 2 

60 echo "IP address of sender: ${sender[1]}" 

61 # Second field of "Received" line S 

62 

63 ÍE3 

64 

65 

66 done < "USD" Ge Recirect Steclowic (ur rile into Loop. 

G7 j 

68 

69 

"0 mbox grep "SI" 4 Send mailbox file to function. 

Jit 

U2 exit S? 

73 

74 # Exercises: 

VS up oc———————— 

76 # 1) Break the single function, above, into multiple functions, 

77 #+ for the sake of readability. 

78 # 2) Add additional parsing to the script, checking for various keywords. 
TS 

80 

81 

82 $ mailbox grep.sh scam mail 

83 MESSAGE of Thu, 5 Jan 2006 08:00:56 -0500 (EST) 

84 IP address of sender: 196.3.62.4 


Exercise: Find other uses for here strings, such as, for example, feeding input to dc. 


Prev Home Next 
Globbing Up I/O Redirection 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Next 


Chapter 20. I/O Redirection 


There are always three default files [1] open, st din (the keyboard), st dout (the screen), and stderr 
(error messages output to the screen). These, and any other open files, can be redirected. Redirection simply 
means capturing output from a file, command, program, script, or even code block within a script (see 
Example 3-1 and Example 3-2) and sending it as input to another file, command, program, or script. 


Each open file gets assigned a file descriptor. [2] The file descriptors for stdin, stdout, and stderr are 
0, 1, and 2, respectively. For opening additional files, there remain descriptors 3 to 9. It is sometimes useful to 
assign one of these additional file descriptors to stdin, stdout, or stderr as a temporary duplicate link. 
[3] This simplifies restoration to normal after complex redirection and reshuffling (see Example 20-1). 


iL COMMAND OUTPUT > 

2 Redirect stdout to a file. 

3 Creates the file if not present, otherwise overwrites it. 

4 

E ig =I & e reer libet 

6 Creates a file containing a listing of the directory tree. 

7 

8 > filename 

9 The > truncates file "filename" to zero length. 

10 If file not present, creates zero-length file (same effect as 'touch'). 
Aral The : serves as a dummy placeholder, producing no output. 

12 

3 > filename 

4 The > truncates file "filename" to zero length. 

5 If file not present, creates zero-length file (same effect as 'touch'). 
6 (Same result as ": »", above, but this does not work with some shells.) 
7 

18 COMMAND_OUTPUT >> 

18 Redirect stdout to a file. 
20 Creates the file if not present, otherwise appends to it. 
Bilt 
22 
23 Single-line redirection commands (affect only the line they are on): 
24 
25 
26 1>filename 
27 Redirect stdout to file "filename." 
28 1>>filename 
29 Redirect and append stdout to file "filename." 
30) 2>filename 
Si Redirect stderr to file "filename." 
32 2>>filename 
33 Redirect and append stderr to file "filename." 
34 &>filename 
35) Redirect both stdout and stderr to file "filename." 
36 This operator is now functional, as of Bash 4, final release. 
37 
38 M>N 
E 4 "M" is a file descriptor, which defaults to 1, if not explicitly set. 
40 # "N" is a filename. 

AT # File descriptor "M" is redirect to file "N." 

42 M>&N 

43 go UMA die zi dalle glescuijEox, Wnion Gees io li. Wiz mt GE 

44 # "N" is another file descriptor. 

45 

46 # 

47 


48 # Redirecting stdout, one line at a time. 


49 
50 
Sil 
52 
59 
54 
55 
56 
S7 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 


(gor | yoo, (nb GSS (69 [NS [9 Gp We) (or. =a) foxy Gal dE Go IS) dE 


Mop p opp ppppÀ|GÀG: 
O (o 00 -1 O Qi i (). M PS. O xo 0 -1 O O1 4 CQ Io E 
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LOGFILE-script.log 


echo "This statement is sent to the log file, \"SLOGFILE\"." 1>SLOGFILE 
echo "This statement is appended to \"SLOGFILE\"." 1>>$LOGFILE 
echo "This statement is also appended to \"SLOGFILE\"." 1>>SLOGFILE 


echo "This statement is echoed to stdout, and will not appear in \"SLOGFIL 


# These redirection commands automatically "reset" after each line. 


Redirecting stderr, one line at a time. 
ERRORFILE-script.errors 


bad commandl1 2>SERRORFILE # Error message sent to SERRORFILE. 
bad command2 2>>SERRORFILE # Error message appended to SERRORFILE. 
bad_command3 # Error message echoed to stderr, 

#+ and does not appear in SERRORFILE. 


# These redirection commands also automatically "reset" after each line. 


2>6&1 
# Redirects stderr to stdout. 
# Error messages get sent to same place as standard output. 
>>filename 2>&1 
bad_command >>filename 2>&1 
# Appends both stdout and stderr to the file "filename" 
2>&1 | [command(s) ] 
bad command 2>&1 | awk '{print $5)' # found 
# Sends stderr through a pipe. 
# |& was added to Bash 4 as an abbreviation for 2»&. 


i»&j 
# Redirects file descriptor i to j. 
# All output of file pointed to by i gets sent to file pointed to by j. 


2&j 
# Redirects, by default, file descriptor 1 (stdout) to j. 
# All stdout gets sent to file pointed to by j. 


0< FILENAME 
« FILENAME 
# Accept input from a file. 
# Companion command to ">", and often used in combination with it. 
# 
# grep search-word <filename 


[j3]<>filename 
# Open file "filename" for reading and writing, 
#+ and assign file descriptor "j" to it. 
# If "filename" does not exist, create it. 
# If file descriptor "j" is not specified, default to fd 0, stdin. 
# 
# An application of this is writing at a specified place in a file. 
echo 1234567890 > File # Write string to "File". 


exec 3<> File # Open "File" and assign fd 3 to it. 
read -n 4 <&3 # Read only 4 characters. 

echo -n . >&3 # Write a decimal point there. 

exec 3>&- # Close fd 3. 

cat File # ==> 1234.67890 


# Random access, by golly. 


EA 


27 # Pipe. 

28 # General purpose process and command chaining tool. 

29) # Similar to ">", but more general in effect. 

30 # Useful for chaining commands, scripts, files, and programs together. 
Su Ge “jis | sore | umie > result—iile 

32 # Sorts the output of all the .txt files and deletes duplicate lines, 
33) # finally saves results to "result-file". 


Multiple instances of input and output redirection and/or pipes can be combined in a single command line. 


1 command « input-file > output-file 
2 
3 commandl | command2 | command3 » output-file 


See Example 16-31 and Example A-14. 


Multiple output streams may be redirected to one file. 


l Jug -y7 > Commence, le; 2b 

2 Capture result of illegal options "yz" in file "command.log." 
3 Because stderr is redirected to the file, 

4 #+ any error messages will also be there. 

E 

6 Note, however, that the following does *not* give the same result. 
7 le =yz Z&i >> commence! loe 

8 Outputs an error message and does not write to file. 

3 

10 If redirecting both stdout and stderr, 

11 #+ the order of the commands makes a difference. 


Closing File Descriptors 


n<&- 

Close input file descriptor n. 
O<&-, <&- 

Close stdin. 
n>&- 

Close output file descriptor n. 
1>&-, >&- 

Close stdout. 


Child processes inherit open file descriptors. This is why pipes work. To prevent an fd from being inherited, 
close it. 


1 # Redirecting only stderr to a pipe. 

2 

3 exec 3>&1 # Save current "value" of stdout. 

“is M-e Ses Sx [ ee bad Se i Close ie 9 tor "gue" (ew met "lg"). 

5 # aa eic 

6 exec 3>&- # Now close it for the remainder of the script. 
3 


8 # Thanks, S.C. 
For a more detailed introduction to I/O redirection see Appendix E. 


20.1. Using exec 


An exec «filename command redirects st din to a file. From that point on, all st din comes from that file, 
rather than its normal source (usually keyboard input). This provides a method of reading a file line by line 
and possibly parsing each line of input using sed and/or awk. 


Example 20-1. Redirecting stdin using exec 


#!/bin/bash 
# Redirecting stdin using 'exec'. 


+ 


exec 6<&0 Link file descriptor #6 with stdin. 


# Saves stdin. 


exec < data-file # stdin replaced by file "data-file" 


+ 


read a2 a o Recle second line of file Wcarca-t dila, m 


echo 

echo "Following lines read from file." 
cho " " 
echo Şal 

echo $a2 


Ji 
2 
3 
4 
5 
6 
y 
8 
9 
10 read a1 Reads itisic lime (ue ile Moarta ide. 
J 
2 
3 
4 
5 
6 
7 
8 


19 echo; echo; echo 


21 exec 0<&6 6«&- 
22 # Now restore stdin from fd #6, where it had been saved, 


23 #+ and close fd #6 ( 6«&- ) to free it for other processes to use. 
24 # 

25 4 «&6 6«&- also works. 

26 


27 echo -n "Enter data " 

28 read b1 # Now "read" functions as expected, reading from normal stdin. 
29 echo "Input read from stdin." 

30 echo " ul 

31 echo "bi = pglu 


33 echo 


35 eri ( 


Similarly, an exec >filename command redirects stdout to a designated file. This sends all command 
output that would normally go to stdout to that file. 


exec N » filename affects the entire script or current shell. Redirection in the PID of the script or shell 
from that point on has changed. However... 


N > filename affects only the newly-forked process, not the entire script or shell. 


Thank you, Ahmed Darwish, for pointing this out. 


Example 20-2. Redirecting stdout using exec 


#!/bin/bash 
# reassign-stdout.sh 


LOGFILE=logfile.txt 


exec 6>&1 # Link file descriptor #6 with stdout. 
# Saves stdout. 


7] 


exec > SLOGFILE # stdout replaced with file "logfile.txt". 


# aoa 2 ui pts # 
# All output from commands in this block sent to file SLOGFILE. 


Seino n tror eniin 
date 

cao; H 

echo 


Scao VOutoue (uc \Wils -al\Y Croriivetayell! 
echo 

iog ani 

echo; echo 

echo "Output of \"df\" command" 
echo 

df 


Mop pop p ppppÀDGÀG:! 
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exec 1>&6 6>&- # Restore stdout and close file descriptor #6. 


echo 

echo "== stdout now restored to default == 
echo 

le. ced 

echo 


CO CO CO CO CO CO CO CO DNO 
mdp on (sp ge ©) [Sos qp «€» 9 


eve (0) 


Example 20-3. Redirecting both stdin and stdout in the same script with exec 


1 #!/bin/bash 

2 upperconv.sh 

3 Converts a specified input file to uppercase. 

4 

5 E FILE ACCESS-70 

6 E WRONG ARGS-71 

7 

Ba qp EE E I] # Is specified input file readable? 
9 then 
10 echo "Can't read from input file!" 
all echo "Usage: $0 input-file output-file" 

12 esit ug MUH JNCICINSS 

PRF # Will exit with same error 
14 #+ even if input file ($1) not specified (why?). 
15 

JG. aie [| — WSZ i] 

17 then 

18 echo "Need to specify output file." 

Lg echo "Usage: $0 input-file output-file" 
20 exit $E WRONG ARGS 
Zl seal 


N 
N 


23 
24 
2/5 
26 
Zi) 
28 
29 
30 
Su 
32 
33 
34 
39 
36 
37 
38 
3 
40 
41 
42 
43 
44 
45 


exec 4«&0 
exec « $1 # Will read from input file. 


exec 7>&1 
exec » $2 # Will write to output file. 


# Assumes output file writable (add check?). 

# 

Ga — | itr sx AE # Uppercase conversion. 
# NEES # Reads from stdin. 
# QC # Writes to stdout. 
# However, both stdin and stdout were redirected. 
# Note that the 'cat' can be omitted. 
# 
exec 1>&7 7>&- # Restore stout. 
exec 0<&4 4<&- # Restore stdin. 
# After restoration, the following line prints to stdout as expected. 


echo "File \"S1\" written to \"$2\" as uppercase conversion." 


exe O 


I/O redirection is a clever way of avoiding the dreaded inaccessible variables within a subshell problem. 


Example 20-4. Avoiding a subshell 


NPRPRPP PPP PP 
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#!/bin/bash 
# avoid-subshell.sh 
# Suggested by Matthew Walker. 


Lines=0 
echo 
cat myfile.txt | while read line; 
ao. T 
echo $line 
(( Lines++ )); 4$ Incremented values of this variable 
#+ inaccessible outside loop. 
# Subshell problem. 
} 
done 
echo "Number of lines read = $Lines" # 0 
# Wrong! 
Chg W n 
exec 3«» myfile.txt 
while read line <&3 
do 4 
echo "Sline" 
(I mme E #  Incremented values of this variable 
#+ accessible outside loop. 
# No subshell, no problem. 
} 
done 
exec 3>&- 


echo "Number of lines read = $Lines" # 8 


37 echo 

39 exit 0 

41 # Lines below not seen by script. 
AS f cat myEile TxE 


45 Line 
46 Line 
47 Line 
48 Line 
49 Line 
50 Line 
51 Line 
(5:21 abes 


GO. = oy, OT d 69 IND CES 


Notes 


[1] By convention in UNIX and Linux, data streams and peripherals (device files) are treated as files, in a 
fashion analogous to ordinary files. 


[2] A file descriptor is simply a number that the operating system assigns to an open file to keep track of it. 
Consider it a simplified type of file pointer. It is analogous to a file handle in C. 


[3] Using file descriptor 5 might cause problems. When Bash creates a child process, as with 
exec, the child inherits fd 5 (see Chet Ramey's archived e-mail, SUBJECT: RE: File descriptor 5 is held 
open). Best leave this particular fd alone. 
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20.2. Redirecting Code Blocks 


Blocks of code, such as while, until, and for loops, even if/then test blocks can also incorporate redirection of 
stdin. Even a function may use this form of redirection (see Example 24-11). The « operator at the end of 
the code block accomplishes this. 


Example 20-5. Redirected while loop 


#!/bin/bash 
# redir2.sh 


T 
2 
3 
“aie [| See US i] 
5 then 
6 Filename=names.data # Default, if no filename specified. 
7 else 
8 Filename-$1 
9 fi 
10 #+ Filename=${1:-names.data} 
11 # can replace the above test (parameter substitution). 
12 
13 count=0 
14 
15 echo 
16 
17 while [ "$name" != Smith ] # Why is variable $name in quotes? 
18 do 
19 read name # Reads from $Filename, rather than stdin. 
20 


echo $name 


Al ler Sooitine m I4 

22 done <"$Filename" # Redirects stdin to file $Filename. 
23 # OE quae 

24 

25 echo; echo "Scount names read"; echo 

26 

27 exit 0 

28 


Note that in some older shell scripting languages, 
* the redirected loop would run as a subshell. 
Therefore, $count would return 0, the initialized value outside the loop. 
Bash and ksh avoid starting a subshell *whenever possible*, 
+ so that this script, for example, runs correctly. 
(Thanks to Heiner Steven for pointing this out.) 


However 
Bash *can* sometimes start a subshell in a PIPED "while-read" loop, 
+ as distinct from a REDIRECTED "while" loop. 


C9 CO CO CO CO CO CO CO CO CO DN 
© co Soy ce Ww ES COL CO, 


A 
e 


abc-hi 
echo -e "1\n2\n3" | while read 1 
do abc="S$1" 
echo $abc 
done 
echo $abc 


oe 
ps 


2 Pb LS a ue 
sj ey (Ga) dex ($9. Je») 


# Thanks, Bruno de Oliveira Schneider, for demonstrating this 
48 #+ with the above snippet of code. 
49 # And, thanks, Brian Onn, for correcting an annotation error. 


Example 20-6. Alternate form of redirected while loop 
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!/bin/bash 


This is an alternate form of the preceding script. 


Suggested by Heiner Steven 
+ as a workaround in those situations when a redirect loop 
+ runs as a subshell, and therefore variables inside the loop 
+do not keep their values upon loop termination. 


stie dp mw. WSL ] 
then 
Filename-names.data # Default, if no filename specified. 
else 
Filename-$1 
ít 


exec 3<&0 # Save stdin to file descriptor 3. 
exec 0<"SFilename" # Redirect standard input. 


count=0 
echo 


while [ "Sname" != Smith ] 
do 
read name # Reads from redirected stdin ($Filename). 
echo $name 
ies. “Wicveiblinte. sr LU 
done # Loop reads from file $Filename 
#+ because of line 20. 


# The original version of this script terminated the "while" loop with 
#+ done <"SFilename" 

# Exercise: 

# Why is this unnecessary? 


exec 0<&3 # Restore old stdin. 
exec 3«&- # Close temporary fd 3. 
ceho acing “"Se@owinc memes cece ecko 

aie (0) 


Example 20-7. Redirected until loop 


(E a <a) (ox web HES Ger [9 [5 
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#!/bin/bash 
# Same as previous example, but with "until" loop. 


aie [ =2 “Si! ] 
then 
Filename=names.data # Default, if no filename specified. 
else 
Filename=$1 
Tan 


# while [ "$name" != Smith ] 
mmcal | Vemana” = Simaeley ] # Change != to =. 


19. Clo) 

14 read name # Reads from $Filename, rather than stdin. 
15 echo $name 

16 done <"SFilename" # Redirects stdin to file $Filename. 

17 # SS RS SARIS E 

18 

19 # Same results as with "while" loop in previous exampl 

20 

Ail «eae. O 


Example 20-8. Redirected for loop 


#!/bin/bash 


lett eae t. SEEN] 
then 
Filename=names.data # Default, if no filename specified. 
else 
Filename=$1 
i3 
line count-' wc $Filename | awk '{ print $1 jJ" 
# Number of lines in target file. 
# 


# Very contrived and kludgy, nevertheless shows that 
#+ it's possible to redirect stdin within a "for" loop... 
#+ if you're clever enough. 


NPRPRPPRP PPP PY 
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# 
# More concise is line _count=$ (we -1 < "SFilename") 
for name in 'seq $line count' Recall that "seq" prints sequence of numbers. 
21 # while [ "$name" != Smith ] == more complicated than a "while" loop == 
22. CO 
ei read name Reads from $Filename, rather than stdin. 
24 echo $name 
25 ane dp Snemma = Smich ] Need all this extra baggage here. 
26 then 
21 break 
28 ira 
29 done <"SFilename" # Redirects stdin to file $Filename. 
30 # (S DRIN QN ONG ATO 
31 
32 exit 0 


We can modify the previous example to also redirect the output of the loop. 


Example 20-9. Redirected for loop (both stdin and stdout redirected) 


#!/bin/bash 


aie [ —z Visi || 
then 
Filename=names.data # Default, if no filename specified. 
else 
Filename=$1 
Ea 


«c Wer (ec. SS] fex, E A (NS) (53 


[zs 


Savefile-$Filename.new # Filename to save results in. 


(cor ss] roy, (n des ep) AS) a 


FinalName=Jonah 


line _count= we SFilename | awk 


for name in seq $line count' 


do 

read name 

echo "Sname" 

xs [p "Sumwewae"s = USissingllNenine! ] 

then 

break 

Tu 
Clone < "ees dew > WS be 
# KRAKRAKRAKRAKRAKAAKAAKAAKRAAAAAKRAAAARASN 
ese d) 


Example 20-10. Redirected if/then test 


(9) sal (oxi (ub de Go IS I= Gr Wo» (G9) «4| (x) (On gem Go) IS) IS 


NN DN + 
IS) {= (9. Wo 


#!/bin/bash 


we f a CUR] 
then 

Filename=names.data # Default, 
else 

Filename-$1 
ft 
TRUE-1 
xar [| UST | # if true 
then 


read name 
echo $name 
FLORe nane 


# A^AA^AAAA^AAAAAA 


# Reads only first line of file. 


# Nam 


to terminat 


"read" on. 


"dE qoxeauanE. Sil QU 


# Number of lines in target file. 


# Redirects stdin to file S$Filename, 
and saves it to backup file. 


if no filenam 


and 


abit 


specified. 


also work. 


# An "if/then" test has no way of iterating unless embedded in a loop. 


exit 0 


Example 20-11. Data file names.data for above examples 


[s 
cm oe) Cs) 3) fex Gal des (Qe [esr d 


m 
an 


erer 
JC ON 


Aristotle 
Belisarius 
Capablanca 
Euler 
Goethe 
Hamurabi 
Jonah 
Laplace 
AEOCLY 
Purcell 
Schmidt 
Semmelweiss 
Smith 
Turing 


15 Venn 

16 Wilkinson 

17 Znosko-Borowski 

18 

iS d "miles mes er Cerea Cile deus 

2 s Vieschize2 sla’, Wieecliies sila’, Wrechen Wsch raan sny Vreeke Sn sla”. 


Redirecting the st dout of a code block has the effect of saving its output to a file. See Example 3-2. 


Here documents are a special case of redirected code blocks. That being the case, it should be possible to feed 
the output of a here document into the stdin for a while loop. 


# This example by Albert Siersema 
# Used with permission (thanks!). 


function doesOutput () 
# Could be an external command too, of course. 
# Here we show you can use a function as well. 
{ 

lg -al *. jc | awk “iprint S$5,99]" 


nr=0 # We want the while loop to be able to manipulate these and 
totalSize=0 #+ to be able to s the changes after the while finished. 


while read fileSize fileName ; do 
echo "SfileName is $fileSize bytes" 
let nr++ 
(EISE Sabres | (ieee si Sa ex ede ab Sh. 7S) )) # Os "lex otslSuececerilesuize" 
19 done<<EOF 
20 $(doesOutput) 
21 EOF 


23 ChG Wnr riles jeoiceilliline, SicoralSimes lox 


em 


Prev Home Nex 
TO Redirection Up Applications 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Chapter 20. I/O Redirection Nex 


— 


20.3. Applications 


Clever use of I/O redirection permits parsing and stitching together snippets of command output (see Example 
15-7). This permits generating report and log files. 


Example 20-12. Logging events 


! /bin/bash 

logevents.sh 

Author: Stephane Chazelas. 

Used in ABS Guide with permission. 


Event logging to a file. 
ust be run as root (for write access in /var/log). 


ROOT_UID=0 # Only users with SUID 0 have root privileges. 
. NOTROOT-67 # Non-root exit error. 


t1 


ae | Seno ce UISENOQOHE DODUD T 

then 
echo VMUSIE be HOGI TO mim thors SX." 
exit SE_NOTROOT 


NPRPRPPRP PPP PY 
O (o 0» -1 O Oi i& (0. NM. I. O xo O0 -1 O O1 i8 CQ) IO ES 


if aL 
FD_DEBUG1=3 
21 FD_DEBUG2=4 
22 FD_DEBUG3=5 
23 
24 # === Uncomment one of the two lines below to activate script. === 
25 # LOG_EVENTS=1 
26 # LOG_VARS=1 
2 
28 
29 log() # Writes time and date to log file. 
30 ( 
31 echo "$(date)  $*" >&7 # This *appends* the date to the fil 
32 # 4905 à— eoummermol elo CUTE LOM 
33) # See below. 
34 } 
35 
36 
37 
38 case $LOG LEVEL in 
39 1) exec S»&2 AS Jeu 5> / cles mudil; 
2) exec 3>&2 4>&2 5» /dev/null;; 
3) exec 3>&2 4>&2 DR os 
"| ceo 3» eal A> elu 5> elk 
esac 


FD. LOGVARS-6 

if [[ $LOG VARS l] 

then exec 6>> /var/log/vars.log 

else exec 6» /dev/null # Bury output. 
Eal 


Or £A A434 445 4 am uA 
D Wer (ee) =) fex (ar des te) [ees f SS 


Oo 
[E 


FD. LOGEVENTS-7 

xr [| Sue lwaSSRUS 3] 

then 

54 # exec 7 >(exec gawk '(print strftime(), $0}' >> /var/log/event.log) 
55 # Above line fails in versions of Bash more recent than 2.04. Why? 


on 
N 


U1 
w 


56 exec 7>> /var/log/event.log # Append to "event.log". 
57 log Write time and date. 
58 else exec 7» /dev/null # Bury output. 

59 i 


+ 


61 echo "DEBUG3: beginning" >&${FD_DEBUG3} 
63 ls -1 >&5 2>&4 # commandl >&5 2>&4 


65 echo "Done" # command2 


67 echo "sending mail" >&${FD_LOGEVENTS } 
68 # Writes "sending mail" to file descriptor #7. 


69 

70 

71 exit 0 
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Chapter 21. Subshells 


Running a shell script launches a new process, a subshell. 


Definition: A subshell is a child process launched by a shell (or shell script). 


A subshell is a separate instance of the command processor -- the shell that gives you the prompt at the 
console or in an xterm window. Just as your commands are interpreted at the command-line prompt, similarly 
does a script batch-process a list of commands. Each shell script running is, in effect, a subprocess (child 
process) of the parent shell. 


A shell script can itself launch subprocesses. These subshells let the script do parallel processing, in effect 
executing multiple subtasks simultaneously. 


1 #!/bin/bash 
2 4 subshell-test.sh 
3 
A ( 
5 # Inside parentheses, and therefore a subshell 
6 wide [ i Jj # Endless loop. 
T ele 
8 exeo VSivlosingiil wu 5 a ^ 
9 done 
X9 )) 
iil 
12 # Script will run forever, 
13 #+ or at least until terminated by a Ctl-C. 
14 
15 exit $? 4 End of script (but will never get here). 
16 
47 
18 
iS) New, zua the Scripts 
20 sh subshell-test.sh 
all 
22 And, while the script is running, from a different xterm: 
23 ps -ef | grep subshell-test.sh 
24 
25) WULD 12) IJ) PPID G SIUM iy TIME CMD 
26 500 2096 2502  ( 1426 qs 4 00:00:00 sh subshell-test.sh 
al S500 2699 2696 21 14:26 jore/4 00:00:24 sh subshell-test.sh 
28 
29 iE 
30 
31 Analysis: 
32 PID 2698, the script, launched PID 2699, the subshell. 
33 
34 Note: The "UID ..." line would be filtered out by the "grep" command, 


35 but is shown here for illustrative purposes. 
In general, an external command in a script forks off a subprocess, [1] whereas a Bash builtin does not. For 
this reason, builtins execute more quickly and use fewer system resources than their external command 
equivalents. 


Command List within Parentheses 


( command1; command2; command3; ... ) 
A command list embedded between parentheses runs as a subshell. 


Variables in a subshell are not visible outside the block of code in the subshell. They are not accessible to the 
parent process, to the shell that launched the subshell. These are, in effect, variables local to the child process. 


Example 21-1. Variable scope in a subshell 


MN oL pop pppppGÀ:! 
O (o 00 1 O Ui i& (Q). M I. O «(o 0 «1 O O1 i Q I E 


UON KERNEK UNEDEK 
j— €x» Wer (Ge) =a) jm ab des Gel [eS qp 


CO0 CO CO CO CO CO CO CO 
ey (ger c x) Gal des (eu Iss) 


op 
= €» 


OP PF PF a SP SS us 
© Wey (ey SS) fy Gal des (er fe» 


oul 
N He 


"oni e (On) emos): Tl 
(set c rexy yl des (63) 


59 


#!/bin/bash 
# subshell.sh 


echo 


echo "We are outside the subshell." 

echo "Subshell level OUTSIDE subshell = ŞASE SUBSHELL" 

# Bash, version 3, adds the new SBASH SUBSHELL variable. 
echo; echo 


TJ 


outer variable-Outer 

global variable- 

# Define global variable for "storage" of 
#+ value of subshell variable. 


( 

cho "We are inside the subshell." 

echo "Subshell level INSIDE subshell = SBASH_SUBSHELL" 
inner variable-Inner 


echo "ese amsaicle Sulosineilil, Winner variable Y = Sinner wasuededole 
ceho "Jeuereyw ineice subshell, \Yobleee\ = orter wairalole” 
global_variable="$inner_variable" # Will this allow "exporting" 


#+ a subshell variable? 


echo; echo 


echo "We are outside the subshell." 
echo "Subshell level OUTSIDE subshell = $BASH SUBSHELL" 
echo 
if [| -z "Sinner variabile" ] 
then 

echo "inner variable undefined in main body of shell" 
else 

echo "inner variable defined in main body of shell" 
£1 
echo “Erom maim body Of shell, Winner variable = Sinner variciole” 
# Sinner variable will show as blank (uninitialized) 


#+ because variables defined in a subshell are "local variables". 
# Is there a remedy for this? 
echo "global variable = "$global variable"" # Why doesn't this work? 


echo 


# Additionally 


cho " ie cho 
var=41 # Global variable. 
( let "var+=1"; echo "\Svar INSIDE subshell = $var" ) # 42 
echo "\Svar OUTSIDE subshell = Svar" # 41 
# Variable operations inside a subshell, even to a GLOBAL variable 


60 #+ do not affect the value of the variable outside the subshell! 
61 

62 

63 exit 0 

64 

65 # Question: 

CON MEL SSS 

67 # Once having exited a subshell, 

68 #+ is there any way to reenter that very same subshell 

69 #+ to modify or access the subshell variables? 


See also SBASHPID and Example 33-2. 


Definition: The scope of a variable is the context in which it has meaning, in which it has a value that 
can be referenced. For example, the scope of a local variable lies only within the function, block of code, or 
subshell within which it is defined, while the scope of a global variable is the entire script in which it 
appears. 


c While the $SBASH SUBSHELL internal variable indicates the nesting level of a subshell, the $SSHLVL 
variable shows no change within a subshell. 


1 echo " \SBASH SUBSHELL outside subshell = SBASH SUBSHELL" # 0 
2 ( echo " \SBASH SUBSHELL inside subshell = SBASH SUBSHELL" ) # 1 
3 ( ( echo " NSBASH, SUBSHELL inside nested subshell = $BASH SUBSHELL" ) ) # 2 
4$4*^^ pane Sc CSS OS 

5 

6 echo 

7 

8 echo " \SSHLVL outside subshell = $SHLVL" # 3 

9 ( echo V" \SSRILNWG ameiacls stiskal = SSEIENIEM ) # 3 (No change!) 


Directory changes made in a subshell do not carry over to the parent shell. 


Example 21-2. List User Profiles 


#!/bin/bash 
# allprofs.sh: Print all user profiles. 


# This script written by Heiner Steven, and modified by the document author. 


FILE-.bashrc # File containing user profile, 
Ww wae W o prorilet aua. yieaeplievsl SECTIE. 


for home in ‘awk -F: '{print $6}' /etc/passwd' 
do 
[ -d "$home" ] continue # If no home directory, go to next. 
[ -r "Shome™ ] continue # If not readable, go to next. 
(cd Shome; [ -e SFILE ] && less SFILE) 
done 


# When script terminates, there is no need to 'cd' back to original directory, 
#+ because 'cd Shome' takes place in a subshell. 


Ger (om. <3] O (at dex (93) hs) [5 ce) We (ex c3 yy (nl gES (93. IS) (S 


exu O 


A subshell may be used to set up a "dedicated environment" for a command group. 


1 COMMAND1 
2 COMMAND2 
3 COMMAND3 
& ( 

5) IFS=: 

6 PATH=/bin 
Ji unset TERMINFO 
8 


set -C 
9 Sibsitfst- 5 
10 COMMAND 4 
Jal COMMAND5 
12 exit 3 # Only exits the subshell! 
13 ) 
14 # The parent shell has not been affected, and the environment is preserved. 


15 COMMAND6 
16 COMMAND7 


As seen here, the exit command only terminates the subshell in which it is running, not the parent shell or 
script. 


One application of such a "dedicated environment" is testing whether a variable is defined. 


1 if (set -u; Svariable) 2» /dev/null 

2 then 

3 echo "Variable is set." 

à i3 # Variable has been set in current script, 
5 #+ or is an an internal Bash variable, 
6 
7 
8 


#+ or is present in environment (has been exported). 


# Could also be written [[ S{variable-x} != x || ${variable-y} !- y ]] 
9 4 or [[ ${variable-x} != x$variable Ji 
10 # or [ll Sdpweusssusdlolessx) = cx jj] 
11 Æ or [[ ${variable-x} != x ]] 
Another application is checking for a lock file: 


ie leet Ca 25> foler mrad 


then 


1 > loek C Ia) 

2 

E # lock file didn't exist: no user running the script 
4 else 

5 echo "Another user is already running that script." 
6 exit 65 
Jf Ea 

8 

9 

0 


# Code snippet by Stéphane Chazelas, 
#+ with modifications by Paulo Marcel Coelho Aragao. 


Processes may execute in parallel within different subshells. This permits breaking a complex task into 
subcomponents processed concurrently. 


Example 21-3. Running parallel processes in subshells 


ge lg bier? lists | sore | 
cune de daisies lists | some | 


wie > iisitil23)) & 
wine, > Wasco) € 


Merges and sorts both sets of 
Running in background ensures 


Same effect as 
Gaw lierik ilisie2 lises | 
Gere listä dine list | 


(&x Wey 0O ~-i «ex; Gal dex (9 ISS) p 


en 


# Don't execute the next 


Sort 
Sort 


lists simultaneously. 
parallel execution. 


| vaie > dGuesEl293 & 
| uniq » list456 & 


command until subshells finish. 


al 
12 Glitz dangu123 lisa ac 


Redirecting I/O to a subshell uses the "|" pipe operator, asin 1s -al | (command). 


&") A code block between curly brackets does not launch a subshell. 


{ command]; command2; command3; . . . commandN; } 
1 vari1-23 
2 eho "Seul # 23 
3 
4 4 varl-7b6; ) 
5 echo "Seul # 76 
Notes 


[1] An external command invoked with an exec does not (usually) fork off a subprocess / subshell. 
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Chapter 22. Restricted Shells 


Disabled commands in restricted shells 
. Running a script or portion of a script in restricted mode disables certain commands that would 
otherwise be available. This is a security measure intended to limit the privileges of the script user and 
to minimize possible damage from running the script. 


The following commands and actions are disabled: 


e Using cd to change the working directory. 

e Changing the values of the $PATH, SSHELL, SBASH ENV, or SENV environmental variables. 

e Reading or changing the $SHELLOPTS, shell environmental options. 

* Output redirection. 

e Invoking commands containing one or more /'s. 

* [nvoking exec to substitute a different process for the shell. 

* Various other commands that would enable monkeying with or attempting to subvert the script for an 
unintended purpose. 

e Getting out of restricted mode within the script. 


Example 22-1. Running a script in restricted mode 


#!/bin/bash 


i? Scarcing che serip wien Wye ll Mostio/lovisial 24 
#+ runs entire script in restricted mod 


echo 


echo "Changing directory." 
cd /usr/local 

echo "Now in 'pwd'" 

echo "Coming back home." 
cd 

echo "Now in 'pwd'" 

echo 


# Everything up to here in normal, unrestricted mod 


set -r 
# set restricted has same effect. 
echo "==> Now in restricted mode. <==" 


NPRPRPP PPP PP 
O (o 0» -1 O Ui i (). M PS. O xo 0 -1 O O1 i Q I E 


NO ON 
N e 


echo 
echo 


N N 
B W 


N 
[91] 


echo "Attempting directory change in restricted mode." 
GO os 
exciso) Weill alia ~ joel” Y 


N N 
SRy 


N 
[99] 


echo 
echo 


exeo "SEL = SS tei” 

echo "Attempting to change shell in restricted mode." 
SHELL="/bin/ash" 

echo 

echo "\SSHELL= SSHELL" 


CO CO CO CO CO WWW Dd 
sup cx Ga) des GS [S [5 d» Wo 


38 echo 
3S) echo 


41 echo "Attempting to redirect output in restricted mode." 
412 le ~L /wess/losum > bim. i leg 
43 ls -1 bin.files # Try to list attempted file creation effort. 


45 echo 


AT ex d 
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Chapter 23. Process Substitution 


Piping the stdout of a command into the st din of another is a powerful technique. But, what if you need 
to pipe the stdout of multiple commands? This is where process substitution comes in. 


Process substitution feeds the output of a process (or processes) into the st din of another process. 
Template 


Command list enclosed within parentheses 
»(command list) 


«(command list) 


Process substitution uses / dev / £d/ «n» files to send the results of the process(es) within 
parentheses to another process. [1] 


d There is no space between the the "<" or ">" and the parentheses. Space there would 
give an error message. 


bash$ echo > (true) 
/ dev/£td/63 


bash$ echo «(true) 
/ dev/ £d/ 63 


bash$ echo »(true) «(true) 
/dev/fd/63 /dev/td/62 


bash$ we «(cat /usr/share/dict/linux.words) 
4msszsg dieses aes Jes ioW Gs 


bash$ grep script /usr/share/dict/linux.words | wc 
262 262 3601 


bash$ wc «(grep script /usr/share/dict/linux.words) 
262 262 3601 /dev/fd/63 


$^) Bash creates a pipe with two file descriptors, --f In and fOut--. The stdin of true connects to 
£Out (dup2(fOut, 0)), then Bash passes a /dev/fd/fIn argument to echo. On systems lacking 
/ dev/£d/«n» files, Bash may use temporary files. (Thanks, S.C.) 
Process substitution can compare the output of two different commands, or even the output of different 
options to the same command. 


bash$ comm «(ls -1) «(1s -al) 


coal 12 

eu WE JL DOZO DOZO TS Wer 10 12756 a de) 

EN EN E l bezo loyoweo 4/2. Mer 10 12556 bilez 

EN EW E JL boza leo) LOS war Iie) 12858 1t2.;8lm 
total 20 
drwxrwxrwx 2 9020 DOZO 4095 Ns 10) Lego 
Clee 12 looro bozo A096 isse O 1 7829 os 
=i eye JL bozo bozo 78 Weve 10 12356 iue 
—Y*w-tw-r-- iL bozo BEZO 4/2 wer 10 12356 alex 
Sify = 1 bozo bozo LOS Were 10) 1,2859 2. sila 


Process substitution can compare the contents of two directories -- to see which filenames are in one, but not 
the other. 


JL eite «(Le Sitiesic_chimeccony) «(1s Ssecomel_climaccory) 
Some other usages and uses of process substitution: 


1 read -a list < «( od -Ad -w24 -t u2 /dev/urandom ) 
2 Read a list of random numbers from /dev/urandom, 
3 #+ process with "od" 

4 #+ and feed into stdin of "read" 

E 

6 

T 


From "insertion-sort.bash" example script. 
Courtesy of JuanJo Ciarlante. 


cat <(is —1) 
Same as ber =I | eae 


sort -k 9 <(ls -l /bin) «(ls I /usr/bin) «(ls -1 /usr/X11R6/bin) 
Lists all the files in the 3 main 'bin' directories, and sorts by filename. 
Note that three (count 'em) distinct commands are fed to 'sort'. 


diff <(commandl) «(command2) # Gives difference in command output. 


[Es 


tar (git Tode ex > file ica jovzZ))  GobuseciEguy ene 
Calls wrar (ur Velev iow 8? fSxelbxrexe(puew. mana, incl Worin € > rla car lowe” . 


m= 


Because of the /dev/fd/<n> system feature, 
the pipe between both commands does not need to be named. 


# 
# 
# 
# 
# 
# 


This can be emulated. 


D =a] e Gr des G) [3 [| 603 We) Ge) p (x) (ap des te) I) [3 


= 


# 

Þzip2 e < pipe > file tar:DŅDZz24& 

tar cf pipe $directory name 

rm pipe 

# (Que 

exec 3>&1 

tar cf /dev/fd/4 $directory_name 4>&1 >&3 3>&- | bzip2 -c > file.tar.bz2 3>6- 
exec 3>&- 


INS) 
Sy we 


NNNNN DNDN NY 
=| ey (Gn GSS (68) [S [5 


28 # Thanks, Stéphane Chazelas 
Here is a method of circumventing the problem of an echo piped to a while-read loop running in a subshell. 


Example 23-1. Code block redirection without forking 


1 #!/bin/bash 

2 4 wr-ps.bash: while-read loop with process substitution. 
3 

4 4$ This example contributed by Tomas Pospisek. 
5 # (Heavily edited by the ABS Guide author.) 

6 

7 echo 

8 

9 echo "random input" | while read i 
10 do 
di global-3D": Not available outside the loop." 
12 # ... because it runs in a subshell. 
13 done 
14 


15 echo "\Sglobal (from outside the subprocess) = $global" 
16 # Sglobal (from outside the subprocess) = 


47 

18 echo; Chalet cho 
Jg 

20 while read i 

241. Cle) 


22 echo $i 
23 global-3D": Available outside the loop." 


24 # ... because it does *not* run in a subshell. 
25 done « «( echo "random input" ) 

26 # DUO 

21 

28 echo "\Sglobal (using process substitution) = $global" 
29 4 Random input 

30 4 $global (using process substitution) - 3D: Available outside the loop. 
Sil 

32 

33 echo; echo "#######EFHE"; echo 

34 

33 

36 

37 # And likewise 

38 

39 declare -a inloop 

40 index=0 

41 cat $0 | while read line 

42 do 

4.5 inloop[$index]-2"$1line" 

44 ( (index++) ) 

45 # It runs in a subshell, so 

46 done 

47 echo “OUTPUT = " 

48 echo S${inloop[*] } # ... nothing echoes. 
49 

50 

51 echo; She re cho 

52 

53 

54 declare -a outloop 

55 index=0 

56 while read line 

5/7/ Clo 

58 outloop[$index]="S$line" 

59 ( (index++)) 

60 # It does *not* run in a subshell, so 


Gil clerus < K( esu SO } 

62 echo "OUTPUT = " 

63 echo S{outloop[*] } # ... the entire script echoes. 
64 

$5. era S* 


A reader sent in the following interesting example of process substitution. 


Script fragment taken from SuSE distribution: 


while read des what mask iface; do 

Some commands 

done « «(route -n) 

^ 9 abs. «€ sie. xeexollxexelibgun, SEComel 16 PEO CES SAO VDS EIENEN 


(oer b (x (up Go TIS) [E 


9 To test it, let's make it do something. 
10 while read des what mask iface; do 
11 echo $des $what $mask $iface 


12 done « «(route -n) 

13 

14 Output: 

IS Kernel IP routing table 

16 Destination Gateway Genmask Flags Metric Ref Use Ifac 

17 42.0.00. 9.0.050 255.945.0590 u Q9 © 0 Lo 

18 # 

1$ 

20 As Stéphane Chazelas points out, 

21 #+ an easier-to-understand equivalent is: 

22. TAOS) m | 

23 while read des what mask iface; do # Variables set from output of pipe. 
24 echo $des Swhat $mask Siface 

25 done # This yields the same output as above. 

26 # However, as Ulrich Gayer points out . . . 

27 #+ this simplified equivalent uses a subshell for the while loop, 
2/8) #+ and therefore the variables disappear when the pipe terminates. 
29 

30 # # 

EX 


32 4 However, Filip Moritz comments that there is a subtle difference 
33 #+ between the above two examples, as the following shows. 

34 

39 ( 

36 route -n | while read x; do ((y++)); done 

37 echo Sy # Sy is still unset 

38 

39 while read x; do ((y++)); done « «(route -n) 

40 echo $y 4 $y has the number of lines of output of route -n 


41 

42 

43 More generally spoken 

44 ( 

45 : | x=x 

46 Seems to start a subshell like 
47 : | ( x=x ) 

48 while 


AVG) sse << << (9) 
50 does not 


Sil )) 

52 

53 This is useful, when parsing csv and the like. 

54 That is, in effect, what the original SuSE code fragment does. 
Notes 


[1] This has the same effect as a named pipe (temp file), and, in fact, named pipes were at one time used in 
process substitution. 


— 
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Chapter 24. Functions 


Like "real" programming languages, Bash has functions, though in a somewhat limited implementation. A 
function is a subroutine, a code block that implements a set of operations, a "black box" that performs a 
specified task. Wherever there is repetitive code, when a task repeats with only slight variations in procedure, 
then consider using a function. 


function function_name { 
command... 


) 


or 


function_name () { 
command... 


} 

This second form will cheer the hearts of C programmers (and is more portable). 
As in C, the function's opening bracket may optionally appear on the second line. 
function_name () 


{ 


command... 


} 


8^; A function may be "compacted" into a single line. 


JL eca O { Geno “Has is a deuwoweunibovg 9 Schop } 
2 # $ S 


In this case, however, a semicolon must follow the final command in the function. 


JL deum ©) Jd eeno Vihree is a tuincciem/’s echo ] sp meroni 
2 # p 


Functions are called, triggered, simply by invoking their names. A function call is equivalent to a command. 


Example 24-1. Simple functions 


#!/bin/bash 


JUST_A_SECOND=1 


( # This is about as simple as functions get. 
Sena Vims ies amv EO 


i 
2 
3 
4 
5 romky (0) 
6 
7 
8 echo "Now exiting funky function." 


9 ) # Function declaration must precede call. 
10 
dat 
JL2 xw (() 
13 ( # A somewhat more complex function. 
14 i=0 


1.5) REPEATS-30 


L7 echo 


18 echo "And now the fun really begins." 
3E) echo 

20 

2l Sleep $JUST A SECOND # Hey, wait a second! 
22 while [ $i -lt SREPEATS ] 

29 do 

24 (oko) V! RONGEIONS  ————————-— um 
ZI) Chg ee ARE " 
26 po. We FU EU 
Dial echo 

2219) let "it=1" 

29 done 

3. 

Sil 

32 # Now, call the functions. 

33) 

34 funky 

S5 CUN 

36 

37 exit 0 


The function definition must precede the first call to it. There is no method of "declaring" the function, as, for 
example, in C. 


ie JL 
# Will give an error message, since function "fl" not yet defined. 


declare -f fl # This doesn't help either. 
il # Still an error message. 


# However... 


Seino "Cela ifuwenEibpyer VRANY irom vyaieleubg. Trium enEdLeyan Wied Nur sU) 


eeka Wituuaieiesloim WU EZ NU c 


Mop p opp ppppGÀ: 
O (o 0» 1 O Oi i (Q). M I. O xo 0 -1 O O1 4 CQ IN 
eu 


2L Hil a 6 ineieem "USEZU ais mer eerta y Galilee ual: cling T9930, 
22 #+ although it is referenced before its definition. 

25 # This is permissible. 

24 

DIE # Thanks, S.C. 


£^) Functions may not be empty! 


1 #!/bin/bash 

2 4 empty-function.sh 

E 

4 empty () 

9 1 

e I 

7 

8 exit 0 4 Will not exit here! 
9 
10 4 $ sh empty-function.sh 
11 # empty-function.sh: line 6: syntax error near unexpected token 'J' 


12 
LS 
14 
15 
16 
17) 
18 
19 
20 
2L 
22 
23 
24 
25 
26 
2 
28 


4 empty-function.sh: line 6: 'J' 
# $ echo $? 
# 2 
# However 
not_quite_empty () 
{ 
illegal_command 
} A script containing this function will *not* bomb 
+ as long as the function is not called. 
# Thank you, Thiemo Kellner, for pointing this out. 


It is even possible to nest a function within another function, although this is not very useful. 


(Se) SS) ry) (Onb Hem (63) IS) [35 (9 Wey ee) o] (ox, (Onb ES. Ge) [R3 [3 


«o 


20 


ii (y 
( 
f2 () # nested 
( 
ceho Wewineiesom Ware WW alinesicla Weil YU eU! 
} 
} 
f2 # Gives an error messag 
echo 
iE d 
12. 37 Wow, 3b" s eli saeabeiec "ibo eub U2w, 


# Even a preceding "declare -f £2" wouldn't help. 


# Does nothing, since calling "fl" does not automatically call "f2". 


#+ since its definition has been made visible by calling "fl". 


nr iHeWeugi sr SoCs 


Function declarations can appear in unlikely places, even where a command would otherwise go. 


ls 


E 

the 
b 
{ 


} 
ipa 


bozo. greet 


4 Somet 


NO 


LI 
# I 


=I || toa) 4 echo "toos | s Perales, lo mseless. 

[ SuSE = lowe | 

n 

ozo_greet () # Function definition embedded in an if/then construct. 
echo "Hello, Bozo." 


SNO_EXIT -eq 1 ]] && exit () 


f SNO EXIT is 1, declares 


"exit 


{ true; 


One 


# Works only for Bozo, 


} 


and other users get an error. 


hing like this might be useful in some contexts. 
| EC IPS iL # Will enable function definition below. 


# Function definition in an 


# This disables the "exit" builtin by aliasing it to "true". 


wamo ila sic ., 


24 
25 
26 
2 
28 
2 
30 
St 
32 
33 
34 
39 
36 
Sm 


exit 47 dBahwossx. Wexaic (0) debuoxcabaLode, Not Were O I ETs 


(Que. aabibeuelws 
filename=filel 


-f "$filename" ] && 
tex (0) 4 san -r "Sieilememave echo “taille Ysti lensme” cleilececl. |p 
toe () 4 Gein Warsiile WSseaileioims' iovore, atoyearcl. "7 acowiela loses j 
Foo 


Thanks, S.C. and Christopher Head 


Functions can take strange forms. 


ji 
2 
S 
4 
5 
6 
j 
8 


—(0 t for i in {1..10}; do echo -n "SEUNCNAME"; done; echo; 
2 No space between function name and parentheses. 
This doesn't always work. Why not? 


# 
# 


# Now, let's invoke the function. 
# 


} 


} 


# EOSIN SENS 10 underscores (10 x function name) ! 


# A "naked" underscore is an acceptable function name. 


$^; What happens when different versions of the same function appear in a script? 


il As Yan Chen points out, 

2 when a function is defined multiple times, 
i5) the final version is what is invoked. 

4 This is not, however, particularly useful. 
5 

6 i wine N) 

7 

8 echo “MIWIESIE version Ot iue (9). 

9 

10 

iil cwac (0) 

A 

IL} Cle VUSIS@omel varsom Oi euge O sY 

14 

15 

1G PUNE # Second version of func (). 

Jy 

ig esas SP 

1s) 


20 # It is even possible to use functions to override 
21 #+ or preempt system commands. 
212, sw Oi Cowes, CALS as SNOT chal Sloe. 


24.1. Complex Functions and Function 


Complexities 


Functions may process arguments passed to them and return an exit status to the script for further processing. 


1 function name Sargl Sarg2 


The function refers to the passed arguments by position (as if they were positional parameters), that is, $1, 


$2, and so forth. 


Example 24-2. Function Taking Parameters 


#!/bin/bash 
# Functions and parameters 


DEFAULT=default 


funez i) 4 
3f [ == WW ] 
then 


echo "-Param #1 is \"$1\".-" 
itu 


variable=S {1-SDEFAULT} 


T 
2 
3 
4 
5 
6 
7 
8 
9 
LO else 
T 
2 
3 
4 
5 echo "variable = $variable" 
6 
7 
8 


20 as E WEZ i 

2l then 

272) cho "-Parameter #2 is \"S2\".-" 
BS ipa 


25) return 0 


28 echo 


30 echo "Nothing passed." 
Si PUNEA # Called 
32 echo 


35 echo "Zero-length parameter passed." 
36 tume2 ww # Called 
37 echo 


39 echo "Null parameter passed." 
40 func2 "Suninitialized_param" # Called 
41 echo 


43 echo "One parameter passed." 
44 func2 first # Called with one 
45 echo 


47 echo "Two parameters passed." 
48 func2 first second # Called with two 


cho "-Parameter #1 is zero length. 


# Default param value. 


# Is parameter #1 zero length? 


-" # Or no parameter passed. 


# What does 
#+ parameter substitution show? 


# It distinguishes between 
#+ no param and a null param. 


with no params 


with zero-length param 


with uninitialized param 


param 


params 


49 echo 


50 

SL echo VAIAU \sccond Wapassedum 

BZ, runc Second # Called with zero-length first parameter 
53 echo # and ASCII string as a second one. 

54 

55 exit O0 


(D The shift command works on arguments passed to functions (see Example 35-16). 


But, what about command-line arguments passed to the script? Does a function see them? Well, let's clear up 
the confusion. 


Example 24-3. Functions and command-line args passed to the script 


#!/bin/bash 

# func-cmdlinearg.sh 

# Call this script with a command-line argument, 
#+ something like $0 argl. 


echo usim 


echo Vp3cet Call gro iumcetigos mo ame passec” 
echo "See if command-line arg is seen." 


NPRPRPPRP PPP PY 
O io 00 -1 O Oi i$ (0. NM. IS. O xo 0 -1 O O1 i CQ I E 
~ 


func 
# No! Command-line arg not seen. 
cho Ww " 
echo 
echo "Second call to function: command-line arg passed explicitly." 
21 Euine Si 
22 # Now it's seen! 
23 
24 exit 0 


In contrast to certain other programming languages, shell scripts normally pass only value parameters to 
functions. Variable names (which are actually pointers), if passed as parameters to functions, will be treated 
as string literals. Functions interpret their arguments literally. 


Indirect variable references (see Example 36-2) provide a clumsy sort of mechanism for passing variable 
pointers to functions. 


Example 24-4. Passing an indirect reference to a function 


f!/bin/bash 
# ind-func.sh: Passing an indirect reference to a function. 


de (ES) [m$ qp 


echo var () 
$3 4 

($ echo velu 

79 jj 


message-Hello 
Hello=Goodbye 


echo_var "Smessage" # Hello 

# Now, let's pass an indirect reference to the function. 
echo_var "S{!message}" # Goodbye 

cho LL m P m "n 


# What happens if we change the contents of "hello" variable? 
Hello="Hello, again!" 


io H H H H H H H Eg 
(t3 ior (es. zs] (ex; Ga dev Go o [SS] Wer. (o9) 


echo var "$message" # Hello 
21 echo var "S{!message}" # Hello, again! 
22 
23 exi (0) 


The next logical question is whether parameters can be dereferenced after being passed to a function. 


Example 24-5. Dereferencing a parameter passed to a function 


1 #!/bin/bash 

2 4 dereference.sh 

3 # Dereferencing parameter passed to a function. 
4 4 Script by Bruce W. Clare. 

5 

6 dereference () 

"9 4 

8 y=\s"si" # Name of variable. 

9 echo $y # SJunk 
10 
iil x= eval "expr \"Sy\" "^ 

12 echo $1=Sx 

ILS) eval "S1=\"Some Different Text \"" # Assign new value. 
“lke: 

15 

16 Junk="Some Text" 

17 echo $Junk "before" # Some Text before 

18 

19 dereference Junk 
20 echo SJunk "after" # Some Different Text after 
2s 
22. ae. O 


Example 24-6. Again, dereferencing a parameter passed to a function 


#!/bin/bash 
# ref-params.sh: Dereferencing a parameter passed to a function. 
# (Complex Example) 


ik 
2 
3 
4 
5 ITERATIONS-3 # How many times to get input. 
6 icount=1 
7 
8 my read () { 

E # Called with my read varname, 
WO #+ outputs the previous value between brackets as the default value, 
Lil #+ then asks for a new value. 


15 Tocai leoga var 


14 
L5 
16 
17 
18 
19 
2 
2 
22 
AS 
24 
25 
26 
2 
28 
29 
30 
Su 
32 
33 
34 
35 
36 
37 
38 


echo -n "Enter a value " 
eval "echo m VIS"SIV] VV i: Previews. velie. 
$? eval echo =m “P\SSi] # Easier to understand, 
#+ but loses trailing space in user prompt. 
read local var 
[ e VSULexemll seus | ee eval. SulleWSloeed seus 
+ TAnd lice: e reUlecalmvansethenesetm sd to aes) value. 
} 
echo 
wale | YWSieoeune! =le VSI RATTONS” J 
do 
my_read var 
echo "Entry #S$icount = Svar" 
let "icount += 1" 
echo 
done 
# Thanks to Stephane Chazelas for providing this instructive example. 


exalic d) 


Exit and Return 


exit status 


Functions return a value, called an exit status. This is analogous to the exit status returned by a 
command. The exit status may be explicitly specified by a return statement, otherwise it is the exit 
status of the last command in the function (0 if successful, and a non-zero error code if not). This exit 
status may be used in the script by referencing it as $?. This mechanism effectively permits script 
functions to have a "return value" similar to C functions. 


return 


Terminates a function. A return command [1] optionally takes an integer argument, which is returned 
to the calling script as the "exit status" of the function, and this exit status is assigned to the variable 


$2. 


Example 24-7. Maximum of two numbers 


1 #!/bin/bash 
2 max.sh: Maximum of two integers. 
3 
4 E PARAM ERR-250 4 If less than 2 params passed to function. 
5 EQUAL-251 # Return value if both params equal. 
6 Error values out of range of any 
7 #+ params that might be fed to the function. 
8 
9 max2 () # Returns larger of two numbers. 
TOR # Note: numbers compared must be less than 257. 
i age (p -A Vee | 
2 then 
S return $E PARAM ERR 
á Ea 
5 
@ zi [ "SL" -eg "S2" ] 
7 then 
8 return SEQUAL 


19 else 
2/0 Shae [ pelle Gis Oat ] 


2s then 

22 Terura Sil 
29 else 

24 return SZ 
25 iB ak 

BS seat 

293 } 

28 


AY) web 33 34 

30 return_val=$? 

3 

32 dae [ "Sssbuce valt -eg $1 
33 then 

34 echo "Need to pass two parameters to the function." 

35 Sli | "Sieeiemicin wail? EG SiON, T 

36 then 

37 echo "The two numbers are equal." 

38 else 

3g echo "The larger of the two numbers is $return val." 
4O ita 

41 

42 

43 exit 0 

44 
45 
46 
47 
48 


UM 


|. PARAM ERR ] 


Exercise (easy): 


Convert this to an interactive script, 
+ that is, have the script ask for input (two numbers). 


n 


į ) For a function to return a string or array, use a dedicated variable. 


count_lines_in_etc_passwd() 


[ -r /etc/passwd ]] && REPLY=$ (echo $(wc -1 < /etc/passwd)) 
If /etc/passwd is readable, set REPLY to line count. 
Returns both a parameter value and status information. 

T 'echo' seems unnecessary, but 
+ it removes excess whitespace from the output. 


de db db db — 


10 if count lines in etc passwd 

11 then 

12 echo "There are $REPLY lines in /etc/passwd." 
13 else 

14 echo "Cannot count lines in /etc/passwd." 

Jy sk 


LY sp aanse oio 


Example 24-8. Converting numbers to Roman numerals 


!/bin/bash 
Arabic number to Roman numeral conversion 
Range: 0 - 200 


Jag" Cxcuels, low sic yess. 


Extending the range and otherwise improving the script is left as an exercise. 


We) Xoer | fopy (Gal des (G2) [5r [3 


Usage: roman number-to-convert 


N H H H H H H H H =) T 
(S) Wer (6s) s (ex; (Oa! Wes 1&5 IS) f» 9 


CTR GOR NO ASSESSED E SS RUSO MIND M ESO MES 
i=! (&» We) (es c] (ex; Gal dee GS) [de [5 


C0 CO CO CO CO CO CO CO 
Wer (os c] lx. Gal dev (e9 du» 


op 
i-e» 


LIMIT=200 
E ARG ERR=65 
E OUT OF RANGE-66 
se [| eva US jJ 
then 
echo "Usage: 'basename $0' number-to-convert" 
exit $E ARG ERR 
3E dL 
num-$1 
if [ "$num" -gt SLIMIT ] 
then 
echo "Out of range!" 
exit $E OUT OF. RANGE 
i3 
to roman () # Must declare function before first call to it. 


{ 


number=$1 
factor-$2 


rchar-$3 


let "remainder - number 


whil l 
do 


"Sremainder" 


echo -n $rchar 


let "number -- factor" 


r — number 


let "remaind 


done 


return $number 


# Exercises: 


# So Soe 

# 1) Explain how this function works. 

# Hint: division by successive subtraction. 

# 2) Extend to range of the function. 

# Hint: use "echo" and command-substitution capture. 


U 4A fF Pb SP SP PS aum 
ae Wer oo) —] (ex; Gal dex. (or [n5 


U U1 
N e 


PIT (nb GE (On) G (Oni 
«oO 00 —1 O) oO iS CO 


(Op) Xen» 193» (6m. LORY Ten)» KO) 
(x, (nb dex (63) [n9 [9 SS) 


=| c3] fen X9 £29) 
j— «m We) (ee c3] 


~~ 
w N 


to_roman 
num=$? 
to roman 
num-$? 
to roman 
num-$? 
to roman 
num-$? 
to roman 
num-$? 
to roman 
num-$? 
to roman 
num=$ ? 
to_roman 
num=$ ? 
to_roman 


# Successive calls to conversion function! 
# Is this really necessary??? Can it be simplified? 
echo 


exit 


Snum 


$num 


$num 


$num 


$num 


$num 


$num 


$num 


$num 


100 C 


90 LXXXX 


ay al 


See also Example 11-28. 


The largest positive integer a function can return is 255. The return command is closely tied to 
the concept of exit status, which accounts for this particular limitation. Fortunately, there are 
various workarounds for those situations requiring a large integer return value from a function. 


Example 24-9. Testing large return values in a function 


PRPRPPPRPRPRPHE EB 
ODINDOBWNHFOWODANDOABAWNE 


NO NO PRO INS 
d [RS p «e» 


N N 
ow 


WWNHNDN P2 
H O Ws) (o9) = xen 


re 


ex 


/bin/bash 
return-test.sh 


The largest positive value a function can return is 255. 


turn test () # Returns 


return $1 


whatever passed to it. 


turn test 27 Osk. 
ho $? Returns 27. 
turn test 255 GuESLIDIL ahs. 
so SE Returns 255. 
turn test 257 lduerevoue I 
mo $? Returns 1 (return code for miscellaneous error). 
curn cest —1 0930995 Do large negative numbers work? 
mo SP (Wall cales sectia -15109607 
No! It returns 168. 
Version of Bash before 2.05b permitted 
large negative integer return values. 
Newer versions of Bash plug this loophole. 
This may break older scripts. 
Caution! 
ag O 


A workaround for obtaining large integer "return values" is to simply assign the "return value" to 
a global variable. 


PRPPRPPrRPP RPE 
O0) —1 OY O1 4 (Q0 I9 PB. O i0 0 -1 O) O1 4 CO N H 


Re 


al 
{ 


turn_Val= # Global variable t 
t_return_test () 

fvar-$1 

Return Val-$fvar 

return # Returns 0 (success). 
t return test 1 

ho $? 

si Yrer valus = Skeren Wed" 
t return test 256 

ae rerun valus = SiRerewaeim Well" 
t_return_test 257 

so Ureicuicn valus = Sreicuria val” 


o hold oversize return value of function. 


256 


25 


19 
20 elt return test 25101 
21 echo “raturo valus = Sireiruicin Weil" PASTOL 


A more elegant method is to have the function echo its "return value to stdout," and then 
capture it by command substitution. See the discussion of this in Section 35.7. 


Example 24-10. Comparing two large integers 


#!/bin/bash 
# max2.sh: Maximum of two LARGE integers. 


# This is the previous "max.sh" example, 
#+ modified to permit comparing large integers. 


QUAL=0 # Return value if both params equal. 
.PARAM ERR--99999 4 Not enough params passed to function. 
yt d Out of range of any params that might be passed. 


"op pd 


max2 () 4 "Returns" larger of two numbers. 
{ 
ase [ = WS QU | 
then 
echo $E_PARAM ERR 
return 
iat 


PRPPPPRPRPRPHE EB 
ODIDTOBWNEFOODAINDAOBAWNHE 


dE [ meum -eq TS 2 0 ] 


20 then 

2 echo SEQUAL 

22 return 

23 else 

24 as [ VSI -ge WEZ f 

25 then 

26 retval-$1 

2 else 

28 retval-$2 

29 fg 

30) Ei 

Sal 

34 elu Saucen # Echoes (to stdout), rather than returning value. 
33 # Why? 

Su p 

35 

36 

37 seewEDuem wall=s (xz 33001 3535997) 

38 pe: Function name 

39) (sce. NASSS Params Passer 

40 This is actually a form of command substitution: 

41 #+ treating a function as if it were a command, 

42 #+ and assigning the stdout of the function to the variable "return val." 
43 

44 

45 OUTPUT. 

AG Li “Sreturnaval" Seq VSEEPARAMEERR ] 

47 then 

48 echo "Error in parameters passed to comparison function!" 
49 elif [ "Sreturn_val" -eq "SEQUAL" ] 

50 then 

Sil echo "The two numbers are equal." 

52 else 

53 echo "The larger of the two numbers is Sreturn_val." 


C 
os 


ita 


55 4 
56 
57 exit 0 

58 

59 # Exercises: 

GU 4? eee 

61 # 1) Find a more elegant way of testing 

62 #+ the parameters passed to the function. 

(39) ng 2) Simoliiy the shie/ielovein icwbieiebues) ar VOUTeUT, Y 

64 # 3) Rewrite the script to take input from command-line parameters. 


Here is another example of capturing a function "return value." Understanding it requires some 
knowledge of awk. 


1 month length () # Takes month number as an argument. 

2 # Returns number of days in month. 

S) wagxewElaiDI—w SL 233 Sil 30 Sil SO Sil SL 30 Sil 30 Si” | Declare as local? 
4 echo "“SmomeinD” | aude "f prime SUGAI pa A ically o 

5 S 
6 Parameter passed to function ($1 -- month number), then to awk. 
7 Awk sees this as "print $1 . . . print $12" (depending on month number) 
8 Template for passing a parameter to embedded awk script: 
9 SUS seis ptmparameter ui 

10 

iba Needs error checking for correct parameter rang C=) 

12 #+ and for February in leap year. 

3} 

14 

15 

16 Usage example: 

17 month=4 # April, for example (4th month). 


18 days_in=$(month_length $month) 
19 echo $days in # 30 
20 # 


See also Example A-7 and Example A-37. 


Exercise: Using what we have just learned, extend the previous Roman numerals example to 
accept arbitrarily large input. 


Redirection 


Redirecting the stdin of a function 
A function is essentially a code block, which means its st din can be redirected (as in Example 3-1). 


Example 24-11. Real name from username 


eerren 
Q0 NPO:00-Jo00 50 NpdPD 


#!/bin/bash 

# realname.sh 

# 

# From username, gets "real name" from /etc/passwd. 


ARGCOUNT=1 # Expect one arg. 
E_WRONGARGS=65 


file=/etc/passwd 
pattern-$1 


if [ $4 -ne "SARGCOUNT" ] 
then 


iS) echo "Usage: 'basename $0' USERNAME" 
16 exit $E WRONGARGS 


Jy su 

18 

19 file excerpt () # Scan file for pattern, 

20 { #+ then print relevant portion of line. 
21 while read line # "while" does not necessarily need [ condition ] 
BE do 

23 eho V Sube" || exe Si | amie sre" UT prime S5 LU 

24 # Have awk use ":" delimiter. 

25 done 

AG p sgile p inechlicscit Aint@ PUMECELONY S SicCliuio. 

2. 

28 file excerpt $pattern 

29 

30 # Yes, this entire script could be reduced to 

31 4 Ciaeio DAMMANN ^ ete passwd] eile Se! UI jsxesbene SS) JU 
32 wp or 

33 # awk -F: '/PATTERN/ {print $5)' 

34 # or 

35 # awk -F: '($1 == "username") { print $5 }' # real name from username 
36 4 However, it might not be as instructive. 

37 

38 exit 0 


There is an alternate, and perhaps less confusing method of redirecting a function's st din. This 
involves redirecting the st din to an embedded bracketed code block within the function. 


1 # Instead of: 
2 IWpüoxeicaugum (D) 
3 ( 

4 payee 

DE cle 

6 
J a Wey class 
© Tumetiom (()) 
9 ( 

0 
il Ata 

2 } < file 
3 
4 
5 
6 
yi 


# Similarly, 


JL7! i19360o«eieshexe Q 47 tnis vxoschssi 

Ag 1 

19) { 

20 echo $% 

21 E ox m e 

22 

23 

2 imines Oia (C) # This doesn't work. 
25 1 


26 echo S* 
27 i || tx m Ie) # A nested code block is mandatory here. 


SO s "suo SoG. 


£^; Emmanuel Rouat's sample bashrc file contains some instructive examples of 
functions. 


Notes 
[1] The return command is a Bash builtin. 
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24.2. Local Variables 


What makes a variable local? 


local variables 


A variable declared as local is one that is visible only within the block of code in which it appears. It 
has local scope. In a function, a local variable has meaning only within that function block. 


Example 24-12. Local variable visibility 


func 


{ 


ec 


echo 


NPRPRPPRP PPP PY 
SCOMIDGTHRWNHFOWOMIDUBRWNE 


0 


ho 


echo UNO ese WU 


#!/bin/bash 
# Global and local variables inside a function. 


local loc_var=23 


echo Yi IG vary” 
global_var=999 


# Declared as local variable. 
Pusesitche seca punitin: 


in function = $1loc var" 


# Not declared as local. 
# Defaults to global. 


eene V'xUenlelevedl Suc NU ia ieWhaletcstioi = Eolea verw 
} 
func 
# Now, to see if local variable "loc_var" exists outside function. 


outside function = $loc_var" 


# Sloc_var outside function = 


a # No, Sloc var not visible globally. 
22 echo W\WGiloloall_ vary! OUESICS fumnetion = Sglolbal vart 

23 # S$Sglobal var outside function = 999 
24 4 S$global var is visible globally. 
25 echo 

26 

Ay exile Q 

28 # In contrast to C, a Bash variable declared inside a function 


N 
«o 


ir is local “omy iit cleclarecl as Sel. 


Before a function is called, all variables declared within the function are invisible outside the body 
of the function, not just those explicitly declared as local. 


PRPPRPPR 
Cn 4 w N P O o 0 -1 O OI 4 C) N B 


#!/bin/bash 


func 


{ 


() 


global_var=37 


echo 


func 
echo 


"global_var 


"global_var 


# Visible only within the function block 
#+ before the function has been called. 
# END OF FUNCTION 


= $global var" # global var = 
# Function "func" has not yet been called, 
#+ so $global var is not visible here. 


= $global var" # global var = 37 
# Has been set by function call. 


24.2.1. Local variables and recursion. 


Recursion is an interesting and sometimes useful form of self-reference. Herbert Mayer defines it as"... 
expressing an algorithm by using a simpler version of that same algorithm . . ." 


Consider a definition defined in terms of itself, [1] an expression implicit in its own expression, [2] a snake 
swallowing its own tail, [3] or . . . a function that calls itself. [4] 


Example 24-13. Demonstration of a simple recursive function 


#!/bin/bash 
# recursion-demo.sh 
# Demonstration of recursion. 


RECURSIONS=9 # How many times to recurse. 


T 

2 

3 

4 

5 

6 r_count=0 # Must be global. Why? 

7 

8 recurse () 

I 

0 var="S1" 

T 

2 while [ "Svar" -ge 0 ] 

3 do 

4 echo “Recursion count = "Sr count" +-+ \Svar = "Svar™" 
5 (ht var =— a cw cou bcr Ji 

6 recurse "Svar" # Function calls itself (recurses) 
7 done #+ until what condition is met? 
ig y 

1g 
20 recurse SRECURSIONS 
2 
22 ew. SE 


Example 24-14. Another simple demonstration 


#!/bin/bash 
# recursion-def.sh 
# A script that defines "recursion" in a rather graphic way. 


RECURSIONS=10 
r_count=0 
ep" " 


define recursion () 
{ 
(1 Seount +) 
sp-"$sp"" " 
Sele inl "See 
eeno "WU. Ace WIE ESVE aoo WU # Per 1913 Webster's dictionary. 


while [ $r count -le $RECURSIONS ] 
do 


<=] ep) (Gil PS (93 [m5 [= «59 We (ex <a) ey) (Ont gE (95 de» [3 


18 define recursion 
1g done 

20 ] 

2T 

22 echo 

2/9 echo VWRecuesnieme "V 
24 define recursion 

25 echo 

26 

27 exit S? 


Local variables are a useful tool for writing recursive code, but this practice generally involves a great deal of 
computational overhead and is definitely not recommended in a shell script. [5] 


Example 24-15. Recursion, using a local variable 


!/bin/bash 


factorial 


Does bash permit recursion? 
Well, yes, but... 
mete eo slom tina yot gorca have tocka alin jour Imsecl co coy Ie. 


[AX ARG-5 
E WRONG ARGS-85 
E RANGE ERR-86 


ae qp ew WILY ] 

then 
echo "Usage: 'basename $0! number" 
exit $E WRONG ARGS 

ial 


NPRPRPP PPP PP 
O (o 0» 1 O Ui i& (Q). M IS. O xo 0 -1 O O1 i CQ I E 


NNN 
w N e 


if [ "$1" -gt $MAX ARG ] 
then 
echo "Out of range ($MAX ARG is maximum)." 
# Let's get real now. 
# If you want greater range than this, 
#+ rewrite it in a Real Programming Language. 
exit $E RANGE ERR 
ical 


NO NO NO NY 
=| teny (Oni Ss 


N 
[99] 


revere — (0) 

( 
local number-$1 
# Variable "number" must be declared as local, 
#+ otherwise this doesn't work. 


C9 CO CO CO CO CO CO CO CO CO INO 
(uo) (oer Sa) exp (Oui dex (US SS) [EA e» Ke) 


aie | VSinuimloere!” -e6 © | 
then 
factorial=1 s leweiencsbeul (oue (9 = il, 

40 else 
41 let "decrnum = number JE 
42 fact $decrnum # Recursive function call (the function calls itself). 
43 lee Witeiciconetell = S number = Sew 
44 dE 3L 
45 


46 ieeiebuem Sirecie@res ail 


49) facie Sil 
50 acho Vwaceorial or Sil se SEU 


52 esac 0) 


Also see Example A-15 for an example of recursion in a script. Be aware that recursion is resource-intensive 
and executes slowly, and is therefore generally not appropriate in a script. 


Notes 

[1] Otherwise known as redundancy. 

[2] Otherwise known as tautology. 

[3] Otherwise known as a metaphor. 

[4] Otherwise known as a recursive function. 

[5] Too many levels of recursion may crash a script with a segfault. 


!/bin/bash 


Warning: Running this script could possibly lock up your system! 
If you're lucky, it will segfault before using up all available memory. 


recursive function () 


echo "$1" # Makes the function do something, and hastens the segfault. 
(( Si « S2 3) && wecursive_fumceidem S Sil 4 1 J) Sg 

As long as 1st parameter is less than 2nd, 
+ increment lst and recurse. 


recursive function 1 50000 # Recurse 50,000 levels! 
Most likely segfaults (depending on stack size, set by ulimit -m). 


Recursion this deep might cause even a C program to segfault, 
+ by using up all the memory allotted to the stack. 


HB p|pp|bpÓbpwuHmpmuimnpmnuo 
XO 00 —1 OY O1 4 (0 I9 LPS. O (o O0 1 OY O1 4 CQ N HS 


20) 

2L echo Waals, wili probably not prime,” 

22 exit 0 # This script will not exit normally. 
23 


24 # Thanks, Stéphane Chazelas. 
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24.3. Recursion Without Local Variables 


A function may recursively call itself even without use of local variables. 


Example 24-16. The Fibonacci Sequence 


Mop pop pppppÀ: 
O (o 0» -1 O Oi i5 (0. NM IS. O xo 0 -1 O O1 4 CQ I E 


INSP TS) IS) RY Ror [sS hs) 
SS) cexy a) des eS) je» (5 


CO CO CO CO CO CO CO CO CO CO h2 P2 
We) (eer (ex. Gr ES. Go (MS) [> S&S wo (99 


A A 
ji m 


DA A A A 
yy (On! Wes eS) fs) 


!/bin/bash 

EDORA Fibonacci sequenc 
Author: M. Cooper 

License: GPL3 


(recursive) 


Fibo(0) 0 
Fibo(1) = 1 
else 
madegi(3) = Woe (JIL) s iu99(3-2) 


MAXTERM=15 # Number of terms (+1) to generate. 
MINIDX=2 # If idx is less than 2, then Fibo (idx) 
Fibonacci () 
{ 
idx-$1 # Doesn't need to be local. Why not? 
iit | VSaiebe ake WSN ID | 
then 
Seiho Steht First two terms are 0 1 
else 
e D») Jai 
ceSmmMl=S ( Milgomecek Such ) # Fibo(j-1) 
(4( =aliebz Jy) 3-2 
term2-$( Fibonacci $idx ) # Fibo(j-2) 
echo $(( terml + term2 )) 
4E 3L 


# An ugly, ugly kludge. 


see above. 


= idx. 


# The more elegant implementation of recursive fibo in C 
#+ is a straightforward translation of the algorithm in lines 7 - 10. 


for i in $(seq 0 SMAXTERM) 

do # Calculate SMAXTERM+1 terms. 
igneo (IP aeyesavevereal j$3L) 
Scho =m Wisi aiexo) Ui 


done 

7? OT 23 5 8 is 21 34 55 G9 Ld4 2232 377 Q1 

# Takes a while, doesn't it? Recursion in a script is slow. 
echo 

exit 0 


Example 24-17. The Towers of Hanoi 


1 
2 


#! /bin/bash 
# 


3 The Towers Of Hanoi 
4 Bash script 
5 Copyright (C) 2000 Amit Singh. All Rights Reserved. 
6 http://hanoi.kernelthread.com 
7 
8 Tested under Bash version 2.05b.0(13)-release. 
9 Also works under Bash version 3.x. 
10 
Lal Used in "Advanced Bash Scripting Guide" 
12 #+ with permission of script author. 
13 Slightly modified and commented by ABS author. 
14 
15 # 
16 The Tower of Hanoi is a mathematical puzzle attributed to 
17 #+ Edouard Lucas, a nineteenth-century French mathematician. 
18 
ig There are thr vertical posts set in a base. 
20 The first post has a set of annular rings stacked on it. 
2l These rings are disks with a hole drilled out of the center, 
22 #+ so they can slip over the posts and rest flat. 
ZS) The rings have different diameters, and they stack in ascending 
24 #+ order, according to size. 
255 The smallest ring is on top, and the largest on the bottom. 
26 
2:3) The task is to transfer the stack of rings 
28 #+ to one of the other posts. 
29) You can move only one ring at a time to another post. 
30 You are permitted to move rings back to the original post. 
Sil You may place a smaller ring atop a larger one, 
32 #+ but *not* vice versa. 
319 Again, it is forbidden to place a larger ring atop a smaller one. 
34 
35 For a small number of rings, only a few moves are required. 
36 #+ For each additional ring, 
37 #+ the required number of moves approximately doubles, 
38 #+ and the "strategy" becomes increasingly complicated. 
39 
40 For more information, see http://hanoi.kernelthread.com 
41 #+ or pp. 186-92 of The Armchair Universe by A.K. Dewdney. 
42 
43 
44 TE Vie 
45 Lan] | | 
46 py ePi | | 
Am a | | 
48 | | | | 
49 | | | | ll 
50 | | | | || 
Sl | | | | d 
52 i 
53 | ck ck ck ck ck Ck ck ck ck ck Ck ck KKK KK KKK KKK KKK KK KK KKK KKK KKK ck ck KKK ck ck kk ck ko ck ko Sk ck Mk Sk kv kx ko ok 
54 #1 2 #3 
55 
56 # 
57 
58 
59 E NOPARAM-66 # No parameter passed to script. 
60 E BADPARAM-67 4 Illegal number of disks passed to script. 
61 Moves- # Global variable holding number of moves. 
62 # Modification to original script. 
63 
64 dohanoi() { # Recursive function. 
65 case $1 in 
66 0) 
67 ii 


68 p 


69 domamoi "S((91—1))v S2 S4 $3 

70 echo move $2 "--»" $3 

Til ( (Moves++) ) iy MOCHIELEGEELOM Co Original see c 
12 Gleis. YS((Sil=i1))Y $4 S3 Sz 


74 esac 


77 case $$ in 

78 1) case S(((SiS0)) )) im # Must have at least one disk. 

78) 1) # Nested case statement. 

80 Giolmeuwos Si 1 3 2 

81 echo "Total moves - $Moves" üt Aol 1, where n = # of disks. 
82 exit 0; 

83 P 


85 echo "$0: illegal value for number of disks"; 
86 exit $E BADPARAM; 


91 echo "usage: $0 N" 
92 echo Y Where \"N\" is the number of disks." 
93 exit $E NOPARAM; 


95 esac 


97 4 Exercises: 


1) Would commands beyond this point ever b xecuted? 
Why not? (Easy) 

2) Explain the workings of the workings of the "dohanoi" function. 
(Dak 3E e eUi S the Dewdney reference, above.) 


Se och ch ok 
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Chapter 25. Aliases 


A Bash alias is essentially nothing more than a keyboard shortcut, an abbreviation, a means of avoiding 
typing a long command sequence. If, for example, we include alias Im="'Is -1 | more" in the ~/ .bashrc 
file, then each 1m [1] typed at the command-line will automatically be replaced by a Is -1 | more. This can 
save a great deal of typing at the command-line and avoid having to remember complex combinations of 
commands and options. Setting alias rm="'rm -i" (interactive mode delete) may save a good deal of grief, 
since it can prevent inadvertently deleting important files. 


In a script, aliases have very limited usefulness. It would be nice if aliases could assume some of the 
functionality of the C preprocessor, such as macro expansion, but unfortunately Bash does not expand 
arguments within the alias body. [2] Moreover, a script fails to expand an alias itself within "compound 
constructs," such as if/then statements, loops, and functions. An added limitation is that an alias will not 
expand recursively. Almost invariably, whatever we would like an alias to do could be accomplished much 
more effectively with a function. 


Example 25-1. Aliases within a script 


!/bin/bash 
alias.sh 


shopt -s expand aliases 
Must set this option, else script will not expand aliases. 


Lest, Sona Eemo 
alias Jesse James-'echo "\"Alias Jesse James\" was a 1959 comedy starring Bob Hope."' 
Jesse James 


echo; echo; echo; 


alias Mvis iL 
# May use either single (') or double (") quotes to define an alias. 


ache Ward ailiasecd VULNUS 
11 /usr/X11R6/bin/mk* #* Alias works. 


NPRPRPPRP PPP PY 
SCOMIKDTHRWNHFPOWCMIDGUARWNE 


echo 


N N 
[sy [= 


directory=/usr/X11R6/bin/ 

prefix=mk* # See if wild card causes problems. 

echo "Variables \"directory\" + \"prefix\" = S$directory$prefix" 
echo 


NO PN 
Ae w 


WDT 
YUH wo 


alias lll-"ls -1 Sdirectory$prefix" 


N 
[99] 


echo aree wys eilaleyseye] \ eal S UU 
LLI # Long listing of all files in /usr/X11R6/bin stating with mk. 
# An alias can handle concatenated variables -- including wild card -- o.k. 


CO CO CO CO CO CO CO CO CO CO Dd 
We) (geh | (xy (np dE (3) [SO [5 9» ve 


od 
o 
rey 

Fh 
fs 
E 
iS 
E 


then 
gulag. ee Ie. LU 
echo "Trying aliased \"rr\" within if/then statement:" 
rr /usr/X11R6/bin/mk* #* Error message results! 
# Aliases not expanded within compound statements. 
echo "However, previously expanded alias still recognized:" 
11 /usr/X11R6/bin/mk* 
3E 3L 
echo 
count=0 
while T Scount —lt 3 ] 
do 
alira S rrr ES il 
echo "Trying aliased \"rrr\" within \"while\" loop:" 
rrr /usr/X11R6/bin/mk* #* Alias will not expand here either. 
“7 cues kilns Hyg sexe Command mere ito wisrel 
let count-t-1 
done 
echo; echo 


alias xyz-'cat $0' 4" Serene IGES miese@ulic . 
# Note strong quotes. 
XYZ 
This seems to work, 
F although the Bash documentation suggests that it shouldn't. 


However, as Steve Jacobson points out, 


# 
#4 
# 
# 
#+ the "SO" parameter expands immediately upon declaration of the alias. 


Siw (0) 


The unalias command removes a previously set alias. 


Example 25-2. unalias: Setting and unsetting an alias 


1 #!/bin/bash 
2 # unalias.sh 
3 
4 shopt -s expand aliases # Enables alias expansion. 
5 
6 allies) Jd der db || noray 
3H ibm 
8 
9 echo 
10 
dil wmalias Lim # Unset alias. 
12. IE a 
13 # Error message results, since 'llm' no longer recognized. 
14 
15 exit 0 
bash$ ./unalias.sh 
TOt MEG 
drwxrwxr-x 2 BOZO bozo 3072 Feb 6 14:04 
drwxr-xr-x 40 bozo bozo 2048 Feb 6 14:04 
EWR WIE TR EOZ bozo 199 Feb 6 14:04 unalias.sh 


./unalias.sh: llm: command not found 


Notes 


[1] ..asthe first word of a command string. Obviously, an alias is only meaningful at the beginning of a 
command. 


[21] However, aliases do seem to expand positional parameters. 
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Chapter 26. List Constructs 


The and list and or list constructs provide a means of processing a number of commands consecutively. These 
can effectively replace complex nested if/then or even case statements. 


Chaining together commands 
and list 
1 command-1 && command-2 && command-3 && ... command-n 
Each command executes in turn, provided that the previous command has given a return value of 


true (zero). At the first false (non-zero) return, the command chain terminates (the first command 
returning false is the last one to execute). 


Example 26-1. Using an and list to test for command-line arguments 


#!/bin/bash 


# and list 
mae [Do do c Wiss && echo "Argument #1 = $1" && [ ! -z "$2" ] && \ 
# AN ^^ ^^ 


echo "Argument #2 = $2" 

then 
echo "At least 2 arguments passed to script." 
# All the chained commands return true. 

else 
echo "Fewer than 2 arguments passed to script." 
# At least one of the chained commands returns false. 


£i 
Note that "if [ ! -z $1 ]" works, but its alleged equivalent, 
"if [ -n Si |" does not. 
However, quoting fixes this. 
ap Wf =m VELU JU works, 
S NEC Careful! 


It is always best to QUOTE the variables being tested. 


This accomplishes the same thing, using "pure" if/then statements. 


[3 [S3 [RS Me m eS eS eee rg 
WDG Ae O (er (Ger — dexy (wb des (EX) i [= ©) (Wer Cs c rex) (Gn Ex (es) sy [5 


we [| y em SL ] 
24 then 
25 acho “Argumeme pl = GIU 
2S Teak 
27] xx | 8$ eg "S2w ] 
28 then 


echo "Argument 42 - $2" 
echo "At least 2 arguments passed to script." 


echo "Fewer than 2 arguments passed to script." 


# It's longer and more ponderous than using an "and list". 


(9 CO (9 Ca CO ea CO CO lO 
-) OO» (Q) M FS O 10 
Fh 
H 


amie SP 


Example 26-2. Another command-line arg test using an and list 


!/bin/bash 


ARGS=1 # Number of arguments expected. 
E BADARGS-85 # Exit value if incorrect number of args passed. 


test $4 -ne SARGS && \ 
RAKRAKRAKRKRAANKRAA condition sr 1L 
echo "Usage: 'basename $0' SARGS argument(s)" && exit $E BADARGS 


^^ 


+ then the rest of the lin xecutes, and script terminates. 


Line below executes only if the above test fails. 
echo "Correct number of arguments passed to this script." 


1 

2 

3 

4 

5 

6 

y 

8 

E 

1O If condition #1 tests true (wrong number of args passed to script), 

il 

2 

3 

4 

5 

6 exit 0 
7 
8 


wr to Check exit value, CO a Veche SPW arcen SCTE CSrminat LON, 


Of course, an and list can also set variables to a default value. 


1 argl=S@ && [ -z "Sargl" ] && argl=DEFAULT 

2 

E # Set Sargl to command-line arguments, if any. 

4 # But . . . set to DEFAULT if not specified on command-line. 
or list 

l -command-1 | | command=2- ||| eommand-s3 || ... command-n 


Each command executes in turn for as long as the previous command returns false. At the first true 
return, the command chain terminates (the first command returning true is the last one to execute). 
This is obviously the inverse of the "and list". 


Example 26-3. Using or lists in combination with an and list 


!/bin/bash 


delete.sh, a not-so-cunning file deletion utility. 
Usage: delete filenam 


E BADARGS-85 


alae [var WES 3 
then 
echo "Usage: ~basename $0` filename" 
exit SE_BADARGS # No arg? Bail out. 
else 
file-$1 # Set filename. 
Fi 


| —- euge" ] m Seno "abe WEN invone. oel N 
Cowardly refusing to delete a nonexistent file." 
AND LIST, to giv rror message if file not present. 


Mop p op opp pppÀG: 
O (o 0» -1 O Oi i$ (). M P. O xo 0 -1 O O1 4 CQ I9 


Not cho message continuing on to a second line after an escap 
21 
22 Pa Weren y IT Gem — Sade echo "Ule JY Sirie celera t) 
225 OR LIST, to delete file if present. 
24 
25 Note logic inversion above. 
26 AND LIST executes on true, OR LIST on false. 


2 
28 exit $? 


® If the first command in an or list returns true, it wi 11 execute. 


1 # ==> The following snippets from the /etc/rc.d/init.d/single 

2 #+==> script by Miquel van Smoorenburg 

3 #+==> illustrate use of "and" and "or" lists. 

4 # ==> "Arrowed" comments added by document author. 

E 

6 | =x /ust/bin/eclear | ea /usr/bim/ellearn 

3i 4 ==> Lit /usr/bin/elear exists, thena invoka ie. 

8 # ==> Checking for the existence of a command before calling it 
9 #+==> avoids error messages and other awkward consequences. 

10 

d # ==> 

1,2 

13 # If they want to run something in single user mode, might as well run it... 
14 tor 2 am /ste/rel cl S[O0-—S] [0-9 e cle 

35) # Check if the script is there. 

16 [ =x "Sat T JI comesinns 

L7 # ==> If corresponding file in $PWD *not* found, 

Le #+==> then "continue" by jumping to the top of the loop. 

11$ 

20 # Reject backup files and files generated by rpm. 


case USI shia 
*.rpmsave|*.rpmorig|*.rpmnew|*-|*.orig) 
continue;; 


NNN PN 
ge (ys) mE» pL 


esac 
| Pit = U/ete/elle/SssimgLe" I ce continue 
Set script name, but don't execute it yet. 

Gu tart 


NO NM N 
=a) Gy) (nl 
E 
Il 
ll 
Vv 


28 done 
29 
30 # ==> 


(D The exit status of an and list oranor list is the exit status of the last command executed. 


Clever combinations of and and or lists are possible, but the logic may easily become convoluted and require 
close attention to operator precedence rules, and possibly extensive debugging. 


1 false && true || echo false # false 

2 

3 Same result as 

4 ( false && true ) || echo false # false 

3 BUCENOT 

6 false && ( true || echo false ) # (nothing echoed) 

7 

8 Note left-to-right grouping and evaluation of statements, 
9 #+ since the logic operators "&&" and "||" have equal precedence. 
10 
ial It's usually best to avoid such complexities. 
12 
13 Thanks, S.C. 


See Example A-7 and Example 7-4 for illustrations of using and / or list constructs to test variables. 
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Chapter 27. Arrays 


Newer versions of Bash support one-dimensional arrays. Array elements may be initialized with the 
variable xx] notation. Alternatively, a script may introduce the entire array by an explicit declare -a 
variable statement. To dereference (retrieve the contents of) an array element, use curly bracket notation, 
that is, $ (element [xx] }. 


Example 27-1. Simple array usage 


#!/bin/bash 


area[11]=23 
area[13]=37 
area[51]=UFOs 


# Array members need not be consecutive or contiguous. 


il 
2 
3 
4 
9 
6 
j 
8 
9 
10 
i 
2 
3 
4 
9 
6 
j 
8 


# Some members of the array can be left uninitialized. 
# Gaps in the array are okay. 
# In fact, arrays with sparse data ("sparse arrays") 
#+ are useful in spreadsheet-processing software. 
Ghor n iuar zw 
echo ${area[11]} # {curly brackets} needed. 

ILS) elec) nanea keli = w 

20 echo $[(area[13]) 

all 

22 echo "Contents of area[51] are $[(area[51])." 

23 


24 # Contents of uninitialized array variable print blank (null variable). 
25 echo -n "area[43] = " 

26 echo $(area[43]) 

27 echo "(area[43] unassigned)" 


29 echo 


31 4 Sum of two array variables assigned to third 
32 areal o= esr Sieuesei il} ss Starsa lsi 

33 echo "area[5] = area[11] + area[13]" 

34 echo -n "area[5] = " 

35 echo S{area[5]} 


36 
37 area[6]= expr S{area[11]} + ${area[51]}~ 
38 echo "area[6] = area[11] + area[51]" 


39 echo -n "area[6] = " 
40 echo S$(area[6]) 
41 # This fails because adding an integer to a string is not permitted. 


43 echo; echo; echo 


45 d 
46 # Another array, "area2". 

47 4$ Another way of assigning array variables... 
48 # array name-( XXX YYY ZZZ ... ) 


50 area2-( zero one two three four ) 


Sil 

52 echo -n "area2[0] = " 

53 echo S${area2[0]} 

54 # Aha, zero-based indexing (first element of array is [0], not [1]). 
55 

56 echo -n "area2[1] = " 

57 echo ${area2[1]} # [1] is second element of array. 

58 # 
59) 
60 echo; echo; echo 
61 
62 # 
63 4 Yet another array, "area3". 

64 4 Yet another way of assigning array variables... 
65 4 array name-([xx]-XXX [yy]=YYY ...) 

66 

67 area3=([17]=seventeen [24]-twenty-four) 

68 

69 echo -n "area3[17] 
70 echo S{area3[17] } 
ya 

72 echo -n "area3[24] 
T> eha Silareas [24] } 
74 d 
73 
WG exit 0 


As we have seen, a convenient way of initializing an entire array is the array=( element1 element2 
... elementN ) notation. 


Bash permits array operations on variables, even if the variables are not explicitly declared as arrays. 


1 string-abcABC123ABCabc 

2 echo S{string | @]} abcABC123ABCabc 

3 echo lorrina lh abcABC123ABCabc 

4 echo ${string[0]} abcABC123ABCabc 

5 echo iscrio] No output! 

6 Why? 

7 echo ${#string[@] } i 

8 One element in the array. 
9 The string itself. 

10 


11 # Thank you, Michael Zick, for pointing this out. 
Once again this demonstrates that Bash variables are untyped. 


Example 27-2. Formatting a poem 


1 #!/bin/bash 

2 poem.sh: Pretty-prints one of the ABS Guide author's favorite poems. 
3 

4 Lines of the poem (single stanza). 

5 Line[1]="I do not know which to prefer," 

6 Line[2]="The beauty of inflections" 

7 Line[3]="Or the beauty of innuendoes," 

8 Line[4]="The blackbird whistling" 

9 ime S = 0r jugi eite." 
10 Note that quoting permits embedding whitespace. 


echo 


IR) [ES ee TS Se et 
(c5) ie) Yee) SS} (ox Gi de (9. IS) d 


N N 
N E 


do 


NON 
dy Ww 


done 


NNN 
HUD O1 


do 


N 
[99] 


done 


EPPUR 


echo 


exit 


CO CO CO CO CO CO CO CO CO CO Dd 
© CO =) 6) Cle WD TES oe © 


ws 
SB} 
+ 


itjouue loyeylil 


jowibmqpiE OW 


# Attribution. 
Attrib[1]=" Wallace Stevens" 

Attrib[2]="\"Thirteen Ways of Looking at a Blackbird\"" 
# This poem is in the Public Domain (copyright expired). 


por mees sia il 2 


joueatioieié Wi 


s\n" 


$s\n" 


# Bold print. 


inoue awae aim Jh 2 3 4 & 


# Five lines. 


vetrine ineaz] 


# Two attribution lines. 


"S(Attrib[index])" 


sgr0 # Reset terminal. 
Mejoquie | 


# See 


0 


# Exercise: 


docs. 


odify this script to pretty-print a poem from a text data file. 


Array variables have a syntax all their own, and even standard Bash commands and operators have special 
options adapted for array use. 


Example 27-3. Various array operations 


ecno 
echo 


ecno 


array-( 
# Element 0 i 


#!/bin/bash 
# array-ops.sh: More fun with arrays. 


${array[0] } 
S{array:0} 


S{array:1} 


NPRPRPPRP PPP PY 
COMIDDHRWNHHFOWOAIRDGURWNE 


NNN PN 
as. (X [Sf 


echo 


N N 
OY Ul 


S{#array[0]} 


S{#array} 


S{#array[1]} 


E 


dE dE db db db de HE 
: 
| 


zero one two three four five ) 
2 


4 3 


zero 
zero 

Parameter expansion of first element, 
starting at position # 0 (1st character). 
ero 

Parameter expansion of first element, 
starting at position # 1 (2nd character). 


4 

Length of first element of array. 
4 
Length of first element of array. 
(Alternate notation) 


3 
Length of second element of array. 
Arrays in Bash have zero-based indexing. 


2S echo Sisesexeny [L1 3 # 6 

29 # Number of elements in array. 

30 echo ${#array[@]} # 6 

Bil # Number of elements in array. 

32 

33 echo " " 

34 

35 array2-( [0]="first element" [1]="second element" [3]="fourth element" ) 
36 # A à à 2 A e à P 2 


37 # Quoting permits embedding whitespace within individual array elements. 


39 echo $[array2[0])] ds xix elenent 

40 echo S${array2[1]} # second element 

41 echo $[(array2[2]) # 

42 # Skipped in initialization, and therefore null. 
43 echo ${array2[3]} # fourth element 

44 echo S${#array2[0] } 5 43 (length of first element) 

45 echo ${#array2[*] } # 3 (number of elements in array) 

46 

AT eet 


Many of the standard string operations work on arrays. 


Example 27-4. String operations on arrays 


1 #!/bin/bash 
2 array-strops.sh: String operations on arrays. 
3 
4 Script by Michael Zick. 
5 Used in ABS Guide with permission. 
6 Fixups: 05 May 08, 04 Aug 08. 
7 
8 In general, any string operation using the ${name ... } notation 
9 #+ can be applied to all string elements in an array, 
10) d wita Chs Spese] ass | ex Slama] sosok notatione 
JT. 
1,2 
13 arrayZ-( one two three four five five ) 
14 
15 echo 
16 
Jy) Wiae@lil Lilie; GWIOSie iain; ISRIE SKSie LOM 
18 echo ${arrayZ[@]:0} # one two three four five five 
JL) A All elements. 
20 
21 echo S{arrayZ[@]:1} # two three four five five 
22 a All elements following element [0]. 
23 
24 echo S{arrayZ[@]:1:2} # two three 
25 oy Only the two elements after element[0]. 
26 
27 echo " 1 
28 
29 
30 4 Substring Removal 
3 
32 4 Removes shortest match from front of string(s). 
33 
34 echo S{arrayZ[@]#f*r} # one two three five five 
35 # d # Applied to all elements of the array. 


36 # Matches "four" and removes it. 


3m 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Syl 
52 
53 
54 
55) 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
pel 
72 
218 
74 
1S 
76 
EN 
78 
WE 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
gi 
92 
213) 
94 
9S 
96 
977 
98 
9$; 
100 
ALO ik 
102 


Longest match from front of string(s) 
echo S{arrayZ[@]##t*e} one two four five five 
Me Applied to all elements of the array. 
Matches "three" and removes it. 
Shortest mecca reon back (Qut Sahici he) 
echo S${arrayZ[@]%h*e} one two t four five five 
is Applied to all elements of the array. 
Matches "hree" and removes it. 
Longest match from back of string(s) 
echo $(arrayZ[G]$$t*e) one two four five five 
ae Applied to all elements of the array. 
Matches "three" and removes it. 
cho "n " 
# Substring Replacement 


# Replace first occurrence of substring with replacement. 
echo $(arrayZ[G]/fiv/XYZ) # one two three four XYZe XYZe 
# As # Applied to all elements of the 


# Replace all occurrences of substring. 
echo $(arrayZ[G]//iv/YY) s (ue tuo aree tour IIMS iue 
# Applied to all elements of the 


Delete all occurrences of substring. 

Not specifing a replacement defaults to 'delete' 

echo ${arrayZ[@]//fi/} one two three four ve ve 

RS Applied to all elements of the 


Replace front-end occurrences of substring. 
echo $(arrayZ[G]/4fi/XY) one two three four XYve XYve 
2 Applied to all elements of the 


Replace back-end occurrences of substring. 

echo S${arrayZ[@]/%ve/ZZ} (Oye Ewo Tares eue LAA Ie aLYAZ 
a Applied to all elements of the 

echo $(arrayZ[G]/$0o/XX) one twXX three four five five 
ie Why? 

fou ayo aah " 

replacement () { 

echo m UN ly 
} 
echo ${arrayZ[@]/%e/$ (replacement) } 


# ^ IAN KAAKAAKAKAARAARAANA 

us exei LII Ever lene i oia ipso seat Td 

# The stdout of replacement() is the replacement string. 

# Q.E.D: The replacement action is, in effect, an 'assignment 


cho " " 


# Accessing the "for-each": 
echo S${arrayZ[@]//*/S$ (replacement optional_arguments) } 


d ^^ RARKAKAKRKRAKRAKRAA 
# pat | Lar 1) p dud WAS NAR 
# Now, if Bash would only pass the matched string 


array. 


array. 


array. 


array. 


array. 


103 #+ to the function being called 


104 

105 echo 

106 

107 est © 

108 

109 Before reaching for a Big Hammer -- Perl, Python, or all the rest -- 
110 recall: 

JESESE S( 555 ) 28 Commence Guo LIE TUE LUI c 

12 A function runs as a sub-process. 

ial} A function writes its output (if echo-ed) to stdout. 

114 Assignment, in conjunction with "echo" and command substitution, 
IL. aae can read a function's stdout. 

LAG The name[@] notation specifies (th quivalent of) a "for-each" 
117 #+ operation. 

118 Bash is more powerful than you think! 


Command substitution can construct the individual elements of an array. 


Example 27-5. Loading the contents of a script into an array 


1 #!/bin/bash 

2 # script-array.sh: Loads this script into an array. 

3 # Inspired by an e-mail from Chris Martin (thanks!). 

4 

3) geript conctemes= Sea VSO") ) # Stores contents of this script ($0) 
6 ie 3U8 GUN GCA s 

7 

8 for element in $(seq 0 $((S$(4script contents[80]) - 1))) 

9 do S{#script_contents[@] } 

10 + gives number of elements in the array. 

a 

12 Question: 

13 Why is seq 0 necessary? 

14 Try changing it to seq 1. 

LS echo -n "S{script_contents[$Selement] }" 

le List each field of this script on a single line. 
I7 G7 eebo -mn WS exexcaljee, _icomesmes |[Sllemeine | JL also worka because (Ou Si ooo Jo 
18 echon ma Wi Use Yo —— H ag a riell separator, 

19 done 

20 

21 echo 

22 

23 eet O 

24 

25 # Exercise: 

2G d ——————-- 


27 wp Mocky tois Sees SO ale diens idest 
Axe) Giar dim INES} Owe mel Pormat, 
29 #+ complete with whitespace, line breaks, etc. 


In an array context, some Bash builtins have a slightly altered meaning. For example, unset deletes array 
elements, or even an entire array. 


Example 27-6. Some special properties of arrays 


1 #!/bin/bash 
2 
3 declare -a colors 


# All subsequent commands in this script will treat 
#+ the variable "colors" as an array. 


echo "Enter your favorite colors (separated from each other by a space)." 


4 
5 
6 
7 
8 
9 read -a colors # Enter at least 3 colors to demonstrate features below. 
O0 # Special option to 'read' command, 
11 #+ allowing assignment of elements in an array. 
2 
3 
4 
5 
6 
y 
8 


echo 


element_count=${#colors[@]} 


Special syntax to extract number of elements in array. 
element count-$(4colors[*]) works also. 
Ig) The "@" variable allows word splitting within quotes 
20 #+ (extracts variables separated by whitespace). 
24 
22 This corresponds to the behavior of "$Q" and "$*" 
23 #+ in positional parameters. 
24 
25 index=0 
26 
27 wails [p "iSsuslssxs ihe Vselenenrc Cownt” | 
28 do # List all the elements in the array. 
29 echo S{colors[Sindex] } 
30 ${colors[index]} also works because it's within $( ... } brackets. 
Sil let "index = Sindex + 1" 
32 (Qire e 
319 ((index++) ) 
34 done 
35 Each array element listed on a separate line. 
36 Tie wepe LS) inion clesiliecicl, wis Cae =a WSteoilores |[Saumeles]| i Y 
37 
38 Doing it with a "for" loop instead: 
39 Om a alia “MS eese] pu 
40 do 
41 echo "$i" 
42 done 
43 hank Som) 
44 
45 echo 
46 
47 # Again, list all the elements in the array, but using a more elegant method. 
48 echo S{colors[@] } # echo S{colors[*]} also works. 
49 
50 echo 
5l 
52 # The "unset" command deletes elements of an array, or entire array. 
53 Unser closes Remove 2nd element of array. 
54 Same effect as colors[1]- 
55 echo S${colors[@]} List array again, missing 2nd element. 
56 
57 unset colors Delete entire array. 
58 unset colors[*] and 
2 + unset colors[@] also work. 
60 echo; echo -n "Colors gone." 
61 echo ${colors[@] } List array again, now empty. 
62 
63 exit 0 


As seen in the previous example, either ${array_name[@]} or ${array_name[*]} refers to all the elements 
of the array. Similarly, to get a count of the number of elements in an array, use either ${#array_name[@]} 
or ${#array_name[*]}. ${#array_name} is the length (number of characters) of $(array. name[0]], the first 


element of the array. 


Example 27-7. Of empty arrays and empty elements 


! /bin/bash 
empty-array.sh 


array0= 
arrayl= 
array2= 
array3= 


UT ) # 


) # What 


echo 
ListArray() 
( 
echo 

echo "Elements 
echo "Elements 
echo "Elements 
echo "Elements 
echo 
echo " 
echo " 
echo " 
echo " 
echo 
echo 
echo 
echo 
echo 


Mop pop pop PPP PY 
O (o 00 -1 O Oi i (). NM PB. O xo 0 «1 O O1 iS CQ I E 


in 


NO ON 
Ne 


in 


N 
[99 


in 


N 
A 


in 


NO 
Oo 


‘Ons 
of 
of 
of 


ie JLTE SE 
EESE 
ELESE 
PERSE 


Length 
Length 
Length 
Length 


NNN 
co -—-1 Oo 


C9 CO NO 
| o» «o 


(GE 
of 
of 
of 


elements 
elements 
elements 
elements 


"Number 
"Number 
"Number 
"Number 


w 
N 


) # No elements 


array0: 
arrayl: 
array2: 
array3: 


element 
element 
element 
element 


Eshkenazi, 


Pies SETANE clans }) 
"arrayl" consists of one empty element. 
Ueba yA 3er Gig 


about this array? 


$(arrayO[8]) 
S{arrayl[@]} 
S{array2[@]}" 
S{array3[@] } 


in arrayO = 
arrayl - 
array2 = 


array3 = 


in 
in 
in 
in arrayO = 
arrayl = 
array2 = 
array3 = 


S{#array0 
${#arrayl 
${#array2 
S{#array3 


in 
in 
in 


S{#array0}" 
S{#arrayl}" 
S{#array2}" 
${#array3}" 


Se OS db HE 


Thanks to Stephane Chazelas for the original example, 
+ and to Michael Zick and Omair 
And to Nathan Coulter for clarifications and corrections. 


Sy sor [= (bs) 


for extending it. 


An empty array is not the same as an array with empty elements. 


(Surprise!) 


WWWW WW CO 
OMA -1 O| O1 4 Ww 


ListArray 


Bop 
i €x 


# Adding 
array0=( 
arrayl-( 
array2-( 
array3-( 


"S{arrayO[@]}" 
"S{arrayl[@]}" 
"S{array2[@]}" 
"S{array3[@] 


ListArray 


QP WS bb PS SS 
(Gy We) sy cb von) Gal dex CS) fe») 


on 
[En 


# or 
array0[S{#array0 
arrayl[S{#arrayl 
[ 
[ 


[91 
N 


array2[S{#array2 
array3[S{#array3 


ListArray 


C1 O1 O1 O1 O1 CO! 
co =a] von) 015 C2 


59 # When extended as above 


# Try extending those arrays. 


an element to an array. 


"newl" ) 
"newl" ) 
"newl" ) 
"newl" ) 


, arrays are 'stacks' 


60 
61 


# Above is the 'push' 

# The stack 'height' is: 

height=S$ {#array2[@] } 

echo 

echo "Stack height for array2 = S$height" 


# The 'pop' is: 

unset array2[S{#array2[@]}-1] # Arrays are zero-based, 

height=$ {#array2[@] } #+ which means first element has index 
echo 

echo "POP" 

echo "New stack height for array2 = Sheight" 

ListArray 

# List only 2nd and 3rd elements of arrayO0. 


from-1 # Zero-based numbering. 
to-2 

array3-( S${array0O[@]:1:2} ) 

echo 

echo "Elements in array3: S${array3[@]}" 

# Works like a string (array of characters). 


# Try some other "string" forms. 


# Replacement: 

array4-( S{array0[@]/second/2nd} ) 
echo 
echo 


'Elements in array4: S{array4[@]}" 


# Replace all matching wildcarded string. 
array5-( S{array0[@]//new?/old} ) 


echo 
echo "Elements in array5: S{array5[@]}" 
# Just when you are getting the feel for this 


array6=( S{array0[@]#*new} ) 
echo # This one might surprise you. 
echo "Elements in array6: S${array6[@]}" 


array/=( S{array0[@]#newl} ) 
echo # After array6 this should not be a surprise. 
echo "Elements in array7: S{array7[@]}" 


# Which looks a lot like 

array8-( S{array0[@]/newl/} ) 
echo 
echo "Elements in array8: S{array8[@]}" 


So what can one say about this? 
The string operations are performed on 
+ each of the elements in var[@] in succession. 
Therefore : Bash supports string vector operations. 
If the result is a zero length string, 
* that element disappears in the resulting assignment. 
However, if the expansion is in quotes, the null elements remain. 


Michael Zick: Question, are those strings hard or soft quotes? 
Nathan Coulter: There is no such thing as "soft quotes." 

! What's really happening is that 

ar the pattern matching happens after 

te all the other expansions of [word] 

I+ in cases like ${parameter#word}. 


0. 


126 zap-'new*' 
127 array9-( S${array0[@]/$zap/} ) 


128 echo 

129 echo "Number of elements in array9: S{#array9[@]}" 
130 array9-( "$(array0[80]/$zap/])" ) 

JLSL echo Yalemente in arrays — Sese 9e] gv 

132 4 This time the null elements remain. 

133 echo "Number of elements in array9: S{#array9[@]}" 
134 

135) 


136 # Just when you thought you were still in Kansas 

137 arrayl0=( S${array0[@]#$zap} ) 

138 echo 

139 echo "Elements in arrayl0: S{arrayl10[@]}" 

140 # But, the asterisk in zap won't be interpreted if quoted. 
141 arrayl0-( S${array0[@]#"$zap"} ) 


142 echo 

143 echo "Elements in arrayl0:  $(array10[Q])" 

144 Well, maybe we are still in Kansas : 

145 (Revisions to above code block by Nathan Coulter.) 

146 

147 

148 Compare array7 with arrayl0. 

149 Compare array8 with array9. 

150 

151 Reiterating: No such thing as soft quotes! 

15:2 Nathan Coulter explains: 

1.5/3) Pattern matching of 'word' in S{parameter#word} is done after 
154 #+ parameter expansion and *before* quote removal. 

i55 In the normal case, pattern matching is done *after* quote removal. 
156 

157 Geeaie 


The relationship of ${array_name[@]} and ${array_name[*]} is analogous to that between $@ and $*. This 
powerful array notation has a number of uses. 


Il Copying an array. 

2 much VS euceayyll [T] E )) 

3 (Que 

4 array2="S${arrayl[@]}" 

E 

6 However, this fails with "sparse" arrays, 
7 #+ arrays with holes (missing elements) in them, 
8 #+ as Jochen DeSmet points out. 

9 
10 arrayl[0]- 
LL arrayl[1] Pu assigned 
12 arrayl[2]= 
13} array2-( T ) # Copy it? 
14 

1S Geo Sie 2 [0] } # 0 

16 echo S${array2[2] } # (null), should be 2 
17 # 

18 

1g 
20 
21 # Adding an element to an array. 
22 array-( "S{array[@]}" "new element" ) 
2,9 wp (ue 
24 array[S{#array[*]}]="new element" 
2:5) 


2$ > imus. SoC. 


į ) The array=( element1 element2 ... elementN ) initialization operation, with the help of command 
substitution, makes it possible to load the contents of a text file into an array. 


#!/bin/bash 


filename=sample_fil 


change linefeeds in file to spaces. 
Not necessary because Bash does word splitting, 
+ changing linefeeds to spaces. 


i 

2 

3 

4 

5 # cat sample_file 

6 # 

7 # Labe 

8 # 2 Cl e Gj 

9 

10 

11 declare -a arrayl 

L2 

13 arcayl=( car “Sienileineimne!~ )) # Loads contents 
14 List file to stdout #+ of $filename into arrayl. 
LS 

16 aucicenyl=( eaw UVSirsilbeuvenne! | tee VV Uo y 

JT 

18 

19 


20 

21 echo S${arrayl1[@] } # List the array. 
22 Lal e 2o e io 
23 

24 Each whitespace-separated "word" in the file 
25 #+ has been assigned to an element of the array. 
26 

27 element count-S$(farrayl[*]) 

28 echo $element count # 8 


Clever scripting makes it possible to add array operations. 


Example 27-8. Initializing arrays 


1 #! /bin/bash 
2 array-assign.bash 
3 
4 Array operations are Bash-specific, 
5 #+ hence the ".bash" in the script name. 
6 
7 Copyright (c) Michael S. Zick, 2003, All rights reserved. 
8 License: Unrestricted reuse in any form, for any purpose. 
9 Version: SIDS 
10 
ital Clarification and additional comments by William Park. 
1,2 
1.3 Based on an example provided by Stephane Chazelas 
14 #+ which appeared in an earlier version of the 
15 #+ Advanced Bash Scripting Guide. 
16 
JL) Output format of the 'times' command: 
18 User CPU «space» System CPU 
JL) User CPU of dead children «space» System CPU of dead children 
20 
2L Bash has two versions of assigning all elements of an array 
22 #+ to a new array variable. 
2.9 Both drop 'null reference' elements 
24 #+ in Bash versions 2.04 and later. 
2:5) An additional array assignment that maintains the relationship of 
26 #+ [subscript]-value for arrays may be added to newer versions. 


2 

28 # Constructs a large array using an internal command, 
29 #+ but anything creating an array of several thousand elements 
30 #+ will do just fine. 

Sil 

32 declare -a bigOne=( /dev/* ) # All the files in /dev 
33 echo 

34 echo 'Conditions: Unquoted, default IFS, All-Elements-Of' 
35 echo "Number of elements in array is ${#bigOne[@]}" 

36 

37 4 set -vx 

38 

39 

40 

41 echo 

42 ocho "— = testina =( Siacray(e]} ) = =" 

43 times 

44 declare -a bigTwo-( ${bigOne[@]} ) 

45 # Note parens: A a 

46 times 

47 

48 

49 echo 

50 cho Y= = resting, =S array e] = =! 

51 times 

52 declare -a bigThree=${bigOne[@]} 


5x3) No parentheses this time. 

54 times 

55 

56 Comparing the numbers shows that the second form, pointed out 
57 #+ by Stephane Chazelas, is faster. 

58 

S9 As William Park explains: 

60 #+ The bigTwo array assigned element by element (because of parentheses), 
61 #+ whereas bigThree assigned as a single string. 

62 So, in essence, you have: 

63 Toii. o zs (i D] teen MUT] He cue RI [I] M aot MEN) 
64 Jonixe MBAs s (MON ened: esee UR 

65 

66 Verify this by: echo S${bigTwo[0] } 

67 echo $(bigThree[0]) 

68 

69 

70 I will continue to use the first form in my example descriptions 
71 #+ because I think it is a better illustration of what is happening. 
T 

TS The reusable portions of my examples will actual contain 

74 #+ the second form where appropriate because of the speedup. 

TS 

76 MSZ: Sorry about that earlier oversight folks. 

TT 

78 

TS) Note: 

80 cS 

81 The "declare -a" statements in lines 32 and 44 

82 #+ are not strictly necessary, since it is implicit 

83 #+ in the Array=(... ) assignment form. 

84 However, eliminating these declarations slows down 

85 #+ the execution of the following sections of the script. 

86 Try it, and see. 

87 

88 exit 0 


-&-) Adding a superfluous declare -a statement to an array declaration may speed up execution of subsequent 
operations on the array. 


Example 27-9. Copying and concatenating arrays 


! /bin/bash 
CopyArray.sh 


This script written by Michael Zick. 
Used here with permission. 


How-To "Pass by Name & Return by Name" 
+ or "Building your own assignment statement". 


CpArray Mac() { 
# Assignment Command Statement Builder 


echo -n 'eval ' 


Scho =i "US # Destination name 
Geha m "eq g1" 

acing =m Wu # Source name 

Seino aa EJI yv 


Mop pop opp PPP PY 
O (o 0» -1 O Oi i (). 9. I2. O xo 0 -1 O O1 iS CQ I0 


# That could all be a single command. 
# Matter of style only. 
} 


IS) ISS) ISS) Ber dE» 
Cnt ss (63 PSs) [3 


declare -f CopyArray # Function "Pointer" 
CopyArray-CpArray Mac # Statement Builder 


N N 
-] Oo 


N 
[99] 


Hype () 
{ 


WwW CO N 
[= 1» Wo) 


# Hype the array named $1. 
(Splice it together with array containing "Really Rocks".) 
# Return in array named $2. 


CO 
N 
+ 


local =e MV? 
local -a hype-( Really Rocks ) 


$($CopyArray $1 TMP) 
TMP-( S{TMP[@]} ${hype[@]} ) 
$(SCopyArray TMP $2) 


WWWW WW CO 
OMA -—-1 0) O1 4 Ww 


A e 
(aS 
— 


declar a before-( Advanced Bash Scripting ) 
declar a after 


echo "Array Before = S{before[@]}" 


Hype before after 


Cr fF SF bP bP Sf uS aum au 
(Sy We) (er SS) (om Ga de Co 9 


echo "Array After = S${after[@]}" 


C1 O1 
NS E 


# Too much hype? 


echo "What S{after[@]:3:2}?" 


declar a imoclasic=( Sese[els2s4 Ssses[GeISSs2]9 ) 
# HS=== pubert ing extraction ———— 


ETP (Ont On Grr Gah Gy (Om 
«oO O0 —1 O) O1 iS CO 


echo "Array Modest = ${modest[@]}" 


oO) 
(5) 


61 4 What happened to 'before' ? 


62 
63 echo "Array Befor 


= Sieestesse [e p 7 


64 
65 exit 0 


Example 27-10. More on concatenating arrays 


! /bin/bash 


Copyright (c) 


Michael S. 
Unrestricted reus 


array-append.bash 


Ze. 2003; 


Licens 
Version: 


SIDS 


ey 


NPRPRPPRP PPP PY 
O (o 0» -1 O Oi i$ (). NM P. O xo 0 -1 O O1 4 CQ I E 


Pipe the output of this script to 
+ SO aie lees" E Scroll (gue: Cia ized s 
redirect output to a file. 


declare -a arrayl= 


Array operations are Bash-specific. 
Legacy UNIX /bin/sh lacks equivalents. 


( zerol onel twol ) 


All rights reserved. 
in any form, 


for any purpose. 


Slightly modified in formatting by M.C. 


'more' 


21 4 Subscript packed. 
22 declare -a array2-( [0]-zero2 [2]=two2 [3]=three2 ) 
23 # Subscript sparse -- [1] is not defined. 
24 
25 echo 
26 echo '- Confirm that the array is really subscript sparse. y 
27 echo "Number of elements: 4" # Hard-coded for illustration. 
Ms store (C a = O@ g a « 4 p sw ») 

do 

echo Wilemets [Si] Sieucxes2 [es] 
done 


declare -a dest 


CO CO CO CO CO CO CO CO CO CO Dd 
We) (eor. =a) (ox. Gal dex (Od) [oy lS CS) We) 


] 
} # Strange results, 


# See also the more general code example in basics-reviewed.bash. 


All 


Elements-Of operator' 
subscripts not maintained. 
they are not being dropped. 


possibly a bug. 


# Combine (append) two arrays into a third array. 
echo 
echo 'Conditions: Unquoted, default IFS, 
40 echo '- Undefined elements not present, 
41 # # The undefined elements do not exist; 
42 
43 dest-( ${arrayl[@]} $(array2[0]) ) 
44 4 dest=S{arrayl[@]}${array2[@] 
45 
46 4 Now, list the result. 
47 echo 
48 echo '- - Testing Array Append - -' 
49 cnt=S${#dest [@] } 
50 
51 echo "Number of elements: $cnt" 
52 ior (( t=O ¢ a «& eine pg au D») 
59. Clo 
54 echo "Element [$i]: $(dest[$1i])" 
55 done 


56 

57 # Assign an array to a single array element (twice). 
58 dest[0]-$(array1[GQ]) 

59 dest[1]-$(array2[8]) 

60 

61 # List the result. 

62 echo 

63 Gln "— = Testine) muoobitu3bexel muere — =! 
64 cnt=S${#dest [@] } 

65 

66 echo "Number of elements: S$cnt" 

G7 dq (( x — 0 p a « gu p xem ») 

68 do 

69 echo "Element [$i]: ${dest[$i]}" 
70 done 

VAL 

72 # Examine the modified second element. 
73 echo 

74 echo '- - Reassign and list second element ! 
3/3) 

76 declare -a subArray-$(dest[1]) 

77 cnt=${#subArray[@] } 


78 

79 echo "Number of elements: $cnt" 

GU som (( 2 =O 8 53 < cuu S du D») 

81 do 

82 echo "Element [$i]: ${subArray[$i]}" 

83 done 

84 

85 The assignment of an entire array to a single element 

86 #+ of another array using the '-$( ... )' array assignment 
87 #+ has converted the array being assigned into a string, 

88 #+ with th lements separated by a space (the first character of IFS). 
89 

90 If the original elements didn't contain whitespace 

Sil If the original array isn't subscript sparse 

92 Then we could get the original array structure back again. 
93 

94 Restore from the modified second element. 

95 echo 

96 echo '- - Listing restored element ` 

97 


98 declare -a subArray-( ${dest[1]} ) 
99 cnt=S{#subArray[@] } 


100 

101 echo "Number of elements: S$cnt" 

1492. ser (( mb = © 9 a x youne DO cae 2) 

LOS (lo 

104 echo "Element [$i]: ${subArray[$i]}" 

105 done 

106 echo '- - Do not depend on this behavior. - -' 
107 echo '- - This behavior is subject to change - -' 
108 echo ' in versions of Bash newer than version 2.05b - -' 
LOS 

110 # MSZ: Sorry about any earlier confusion folks. 

iE ala 

132 exe @ 


Arrays permit deploying old familiar algorithms as shell scripts. Whether this is necessarily a good idea is left 
for the reader to decide. 


Example 27-11. The Bubble Sort 


!/bin/bash 
bubble.sh: Bubble sort, of sorts. 


Recall the algorithm for a bubble sort. In this particular version... 


With each successive pass through the array to be sorted, 
+ compare two adjacent elements, and swap them if out of order. 
At the end of the first pass, the "heaviest" element has sunk to bottom. 
At the end of the second pass, the next "heaviest" one has sunk next to bottom. 
Pumgl SO iO EIA. 
This means that each successive pass needs to traverse less of the array. 
You will therefore notice a speeding up in the printing of the later passes. 


exchange () 
{ 
# Swaps two members of the array. 
local temp=${Countries[$1]} # Temporary storage 
#+ for element getting swapped out. 
Countries [$1]=${Countries [$2] } 


NPRPRPPRP PPP PY 
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20 Countries[$2]-$temp 

22 

23 return 

24 ) 

215 

26 declare -a Countries # Declare array, 

27 #+ optional here since it's initialized below. 
28 

29 # Is it permissable to split an array variable over multiple lines 
30 #+ using an escape (M)? 

31 4$ Yes. 

32 

33 Countries=(Netherlands Ukraine Zaire Turkey Russia Yemen Syria \ 
34 Brazil Argentina Nicaragua Japan Mexico Venezuela Greece England \ 
35 Israel Peru Canada Oman Denmark Wales France Kenya \ 

36 Xanadu Qatar Liechtenstein Hungary) 

37 

38 4 "Xanadu" is the mythical place where, according to Coleridge, 

39 #+ Kubla Khan did a pleasure dome decr 

40 

41 

42 clear # Clear the screen to start with. 

43 

44 echo "0: S$(Countries[*])" # List entire array at pass 0. 

45 

46 number_of_elements=${#Countries[@] } 

47 let "comparisons = $number of elements s 

48 

49 count-1 # Pass number. 

50 

Sil wake || "Segunosueisome"- sg  ] # Beginning of outer loop 
52 (olo 

53 

54 index=0 # Reset index to start of array after each pass. 

55 

56 while [ "Sindex" -1t "$comparisons" ] # Beginning of inner loop 
57 do 

58 xi | {Countries Simce] We SfCewuwxiem[ esee Simce se il” jb ] 
59 4 aE OUWNE OIE OIC so 4 

60 # Recalling that \> is ASCII comparison operator 

61 #+ within single brackets. 

62 

63 i? ab l SCout |/Sinaclexx|| = Si(Coyimicienes | ido» Saliockesx c i>)! ql 
64 #+ also works. 


65 then 

66 xchange $index “expr Sindex + 1° # Swap. 
67 ie aL 
68 let vimes q= IL? (Que. index+=1 on Bash, ver. 3.1 or newer. 
69 done # End of inner loop 

70 
p 
72 Paulo Marcel Coelho Aragao suggests for-loops as a simpler altenative. 
73 
74 ter ((( last = Sinibindsysie_ene_ Silene S i 9. lease s 0 2 last = I) 
75 di imate Joy (Qa. IRE A (Thanks!) 
76 do 

33 iene (( ak e 0 p al K dew Boc Jj 

78 do 
79 LIE VS Comins S]j > wS Coume riss S a1») 439 N 
80 && exchange $i $((it1)) 

81 done 

82 done 

83 
84 
85 
86 let "comparisons -- 1" # Since "heaviest" element bubbles to bottom, 

87 #+ we need do one less comparison each pass. 

88 

89 echo 

90 echo "Scount: $í(Countries[80])" # Print resultant array at end of each pass. 
91 echo 

92. lue "exopumE re db # Increment pass count. 

93 

94 done # End of outer loop 

95 # All done. 

96 

97 exit © 


Is it possible to nest arrays within arrays? 


1 #!/bin/bash 

2 # "Nested" array. 

3 

4 # Michael Zick provided this example, 

5 #+ with corrections and clarifications by William Park. 

6 

7 AnArray-( $(1s inod ignore-backups --almost-all \ 

8 directory UL Ue din color=non time=status \ 

9 —-sort=time -1 $(PWD) ) ) # Commands and options. 

10 

Li Spaces are significant . . . and don't quote anything in the above. 
12 

13 SubArray-( S${AnArray[@]:11:1} ${AnArray[@]:6:5} ) 

14 This array has six elements: 

15 #+ SubArray-( [0]=${AnArray[11]} [1]=${AnArray[6]} [2]=${AnArray[7] } 
16 [3]=S{AnArray[8]} [4]=S${AnArray[9]} [5]=S{AnArray[10]} ) 
117 

NEST Arrays in Bash are (circularly) linked lists 

IS) a> Or IES Sieienme, (eneke 9) 5 
20 SO, wais isi aciemeulihy ci imesiecel eunice 
All Gear lone ab ey ieblinCeslLomeullilky Gili Ileus. 
22 


23 echo "Current directory and date of last status change:" 
24 echo "S${SubArray[@]}" 


26 exire (0) 


Embedded arrays in combination with indirect references create some fascinating possibilities 


Example 27-12. Embedded arrays and indirect references 


1 #!/bin/bash 
2 # embedded-arrays.sh 
3 # Embedded arrays and indirect references. 
4 
5 # This script by Dennis Leeuw. 
6 # Used with permission. 
7 # Modified by document author. 
8 
9 
10 ARRAY1= ( 
ili VAR1_1=valuel1l 
12 VAR1, 2-value12 
L3 VAR1, 3-value13 
14 ) 
15 
16 ARRAY2-( 
ly VARIABLE-"test" 
18 STRING="VAR1=valuel VAR2-value2 VAR3=value3" 
19 ARRAY21=$ {ARRAY1[*] } 
20) }) # Embed ARRAY1 within this second array. 
2 
ZA 1tpuaeiEiom eime O 1 
BS OLD_IFS="SIFS" 
24 IFS=$'\n' # To print each array element 
25 #+ on a separate line. 
26 TEST1-"ARRAY2[*]" 
27 local ${!TEST1} # See what happens if you delete this line. 
28 # Indirect reference. 
29 # This makes the components of $TEST1 
30 #+ accessible to this function. 
Sil 
32 
33 # Let's see what we've got so far. 
34 echo 
35 echo STESTI = SWDSSID # Just the name of the variable. 
36 echo; echo 
37 echo "{\STEST1} = S$(!TEST1)" # Contents of the variable. 
38 # That's what an indirect 
39) #+ reference does. 
40 echo 
41 cho ” ins cho 
42 echo 
43 
44 
45 # Print variable 
46 echo "Variable VARIABLE: SVARIABLE" 
47 
48 # Print a string element 
49 TFS—"SOLD_ IFS" 
50 TEST2="STRING[*]" 
5T Local SH Danes y # Indirect reference (as above). 
52 echo "String element VAR2: SVAR2 from STRING" 


# Print an array element 
TEST2-"ARRAY21[*]" 


C1 O1 O1 
ow w 


56 local S n SZ } # Indirect reference (as above). 
57 echo "Array element VAKI 1: $VAR1 1 from ARRAY21" 

5g j 

59 

60 print 

61 echo 

62 

63 exit 0 

64 

65 4 As the author of the script notes, 

66 #+ "you can easily expand it to create named-hashes in bash." 
67 # (Difficult) exercise for the reader: implement this. 


Arrays enable implementing a shell script version of the Sieve of Eratosthenes. Of course, a resource-intensive 
application of this nature should really be written in a compiled language, such as C. It runs excruciatingly 
slowly as a script. 


Example 27-13. The Sieve of Eratosthenes 


!/bin/bash 
Sieve.sh (ex68.sh) 


Sieve of Eratosthenes 
Ancient algorithm for finding prime numbers. 


This runs a couple of orders of magnitude slower 


+ than the equivalent program written in C. 
jOWER. LIMIT-1 i? S ibreists teet Ta GT Waele dle, 
UPPER LIMIT-1000 # Up to 1000. 
(You may set this higher . . . if you have time on your hands.) 
PRIME-1 


NON PRIME-0 


let SPLIT-UPPER LIMIT/2 
# Optimization: 
# Need to test numbers only halfway to upper limit. Why? 


Mop pop opp PPP PY 
O (o 00 -1 O Ui i$ (). M P2. O xo 0 -1 O O1 4 Q Io E 


NON 
NOP 


declar a Primes 
# Primes[] is an array. 


IS) [Sor 1S) 
ow UC 


ijaislie berline. (0) 
( 


# Initialize the array. 


N N 
yA 


N 
[99] 


i-$LOWER LIMIT 
viwe |p WS" ow USOARA MIIN | 
do 
Primes [i]=SPRIM 
hee Val owe pu 
done 
# Assume all array members guilty (prime) 
#+ until proven innocent. 


} 


C9 CO hO 
[> cm» Wo) 


w 
N 


E 


C0 CO CO CO CO CO CO 
O 0 TNA BW 


[i 
e 


print primes () 


41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sib 
52 
55 
54 
95 
56 
5j 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
T3 
WZ 
73 
74 
1/5 
76 
T3) 
78 
Ws 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
21 
22 
9s 
94 
95 
96 
2 
98 
29 
100 
101 
102 
119.5) 
104 
105 
106 


{ 


# Print out the members of the Primes[] array tagged as prime. 


i=SLOWER_LIMIT 


until [ "$i" -gt "SUPPER_LIMIT" ] 
do 
sse | "Sesso" seer VEPRIME ] 
then 
jorealiaigie Ue Sat 
# 8 spaces per number gives nice, even columns. 
iE aL 
iet "i +=" 
done 
} 
sift () # Sift out the non-primes. 
{ 
let i-$LOWER LIMIT-1 


# Let's start with 2. 


Maen [p WS Sere USP OEY T 
do 


mie [p VUSfrreiimes TLI -eer VSPRIMES ] 
# Don't bother sieving numbers already sieved (tagged as non-prime). 
then 


while [ "$t" -le "SUPPER LIMIT" ] 
do 
ete Wie es aL U 
Primes[t]-$NON PRIME 
# Tag as non-prime all multiples. 
done 


ira 


jet Ti t= qu 
done 


# 
# main () 

# Invoke the functions sequentially. 

initialize 

aliti 

print primes 

# This is what they call structured programming. 


107 # # 
108 # Code below line will not execute, because of 'exit.' 


[s 
(&5) 
«o 


# This improved version of the Sieve, by Stephane Chazelas, 
#+ executes somewhat faster. 


fa 
(= 


# Must invoke with command-line argument (limit of primes). 


let SPLIT=UPPER_LIMIT/2 # Halfway to max number. 


PRPRPPrPrRPP PR 


2 
3 
4 
15 UPPER_LIMIT=$1 # From command-line. 
6 
y 
8 


Primes-( '' $(seq SUPPER LIMIT) ) 


AIL -wüskeabl (6 WD ub sem dL y c Ses ar 25) # Need check only halfway. 


126 Mae (4 (ose x p) > RER gum )) 


128 Primes[t]- 
1:29) done 


132 echo ${Primes[*] } 


134 deut GE 


Example 27-14. The Sieve of Eratosthenes, Optimized 


!/bin/bash 

Optimized Sieve of Eratosthenes 

Script by Jared Martin, with very minor changes by ABS Guide author. 
Used in ABS Guide with permission (thanks!). 


Based on script in Advanced Bash Scripting Guide. 
http://tldp.org/LDP/abs/html/arrays.html#PRIMESO (ex68.sh). 


http://www.cs.hmc.edu/~oneill/papers/Sieve-JFP.pdf (reference) 
Check results against http://primes.utm.edu/lists/small/1000.txt 


Necessary but not sufficient would be, e.g., 
((S(sieve 7919 | wc -w) == 1000)) && echo "7919 is the 1000th prime" 


UPPER LIMIT-$[(1:?"Need an upper limit of primes to search.") 


Primes-( '' $(seq ${UPPER_LIMIT}) ) 


IWVOSSSIE al aL TE 


Mop pop pppppogÀ: 
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Primes[i-1]-'' 4$ 1 is not a prime. 
21 until (( (i += 1) > (S(UPPER LIMIT)/i) )) # Need check only ith-way. 
22 do # Why? 
BS se CS marines ica (ab 103) atl] EI 
24 # Obscure, but instructive, use of arithmetic expansion in subscript. 
25 then 
26 timed (( (3x += x ) c SUPPER xw J) 
2 do Primes[t]=; done 
28 3E dL 
29 done 
30 


31 4$ echo ${Primes[*] } 


312 
33 
34 
39 
36 


echo i? Charge co Oriculmel SErIjoE ior qoweexElEY-1913ueEsome; (SW-—Col. chlisjoilayy)) s 
printf "S8d" S(Primes[*]) 
echo; echo 


exalic SP 


Compare these array-based prime number generators with alternatives that do not use arrays, Example A-15, 
and Example 16-46. 


Arrays lend themselves, to some extent, to emulating data structures for which Bash has no native support. 


Example 27-15. Emulating a push-down stack 


#!/bin/bash 
# stack.sh: push-down stack simulation 


# Similar to the CPU stack, a push-down stack stores data items 
#+ sequentially, but releases them in reverse order, last-in first-out. 


BP-100 Base Pointer of stack array. 
Begin at element 100. 


SP-SBP Stack Pointer. 
Ihinaieae lias ave ico "Jowexe'" (secre) oir Gees 


Data= Contents of stack location. 
Must use global variable, 
+ because of limitation on function return range. 


100 Base pointer < Base Pointer 
99 First data item 
98 Second data item 
More data 
Last data item <-- Stack pointer 
declare -a stack 
push() # Push item on stack. 
( 
ase qp ea. USE 7] # Nothing to push? 
then 
Yeturs 
£i 
ei Wise = DW # Bump stack pointer. 
eek [SS] -S1 
Teburn 
} 
pop () # Pop item off stack. 
{ 
Data= # Empty out data item. 


wie | "SSgE" -ee "SIE" | # Stack empty? 


47 
48 
49 
50 
Bil 
52 
5 
54 
55 
56 
Sy 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
ele 
172 
13 
74 
WS 
76 
33 
78 
WS 
80 
81 
82 
83 
84 
3S 
86 
87 
88 
89 
90 
e 
92 
23 
94 
95 
96 
23 
98 
99 
100 
IOR 
102 
103 
104 
105 
106 
107 
108 
109 
110 
i11 
112 


then 


Ea # This also keeps SP from getting past 100, 


return 


#+ i.e., prevents a runaway stack. 


Data-$(stack[$SP]) 
dE Pat ae lit # Bump stack pointer. 
JUNI 


) 


status, report () # Find out what's happening. 


cho " " 


cho "REPORT 


echo "Stack Pointer = $SP" 
acho "üt. joojsecl WU Sem NU gis ime sical.” 


che " 
echo 
} 
# 
# Now, for some fun. 
echo 
# See if you can pop anything off empty stack. 


Pop 
srSgLuscrSpert 


echo 


push garbage 


pop 
status report # Garbage in, garbage out. 
valusl=2 5; push Svaluel 
value2=skidoo; push Svalue2 
value3=LAST; push Svalue3 
pop LAST 
SLODHSSTODOIL 
pop Skidoo 
SUSLUSCPOPOIELt 
pop XS) 
status_report hast-in, first-out! 
# Notice how the stack pointer decrements with each push, 
#+ and increments with each pop. 
echo 
exit 0 

Exercises: 

1) Modify the "push()" function to permit pushing 

+ multiple element on the stack with a single function call. 
2) Modify the "pop()" function to permit popping 
+ multiple element from the stack with a single function call. 


Wis 4» 3) Ackl Griot ielmexelkaseg; (Eg? cians Cirie reed. duele. 

114 4 That is, return an error code, depending on 

115 4 + successful or unsuccessful completion of the operation, 
116 4 + and take appropriate action. 

JL) 

LES s 20) Usine TEMS See. EIS: Gl SibAiciesinG; jerouiaie, 

119 # + write a stack-based 4-function calculator. 


Fancy manipulation of array "subscripts" may require intermediate variables. For projects involving this, 
again consider using a more powerful programming language, such as Perl or C. 


Example 27-16. Complex array application: Exploring a weird mathematical series 


!/bin/bash 
Douglas Hofstadter's notorious "Q-series": 


Qu» = 9(2) ei 
Qe) = Qs = Q1) se Qum = Qa-2))), EOS 102 
This is a "chaotic" integer series with strange 
* and unpredictable behavior. 

The first 20 terms of the series are: 
L233 455 6 6 6 8 & 8 i) 9 WO ail) ld 32 


See Hofstadter's book, _Goedel, Escher, Bach: An Eternal Golden Braid_, 
3/399 diio ARE 


LIMIT-100 # Number of terms to calculate. 
INEWIDTH-20 4 Number of terms printed per line. 


Mop p opp ppppÀ|GÀG: 
O (o 00 1 O O1 i (). NM PS. O xo 0 -1 O O1 4 CQ Io E 


(9 [rib] e # First two terms of series are 1. 
Ak X92] ext 
22 
23 echo 
24 echo "Q-series [SLIMIT terms]:" 
25 eee =m UST A # Output first two terms. 
26 echo — "SUO Y 
22.7) 
28 for ((n=3; n <= S$LIMIT; n**)) # C-like loop expression. 
29. (le # orm] = Olin — O[n-1]] + Q[n = QO[n-2]] ROL 30102) 
SO qr Need to break the expression into intermediate terms, 
31 #+ since Bash doesn't handle complex array arithmetic very well. 
32 
319 let "nl = $n - 1" n= 
34 Fet Vine ces Sig c PHI n-2 
35) 
36 TOS Egor Sia = S Ond 1E n - Q[n-1] 
37 ile Exgoe Sin = SOMA n — Q[n-2] 
39 
ag LOSS TOTO] y Qm = Ofa] 
40 PISS EO DE I Oia = QO[mc-2)|] 
41 
42 Q[n]= expr STO + ST1^ Q[n - Q[n-1]] + Q[n - Q[n-2]] 
43 echo -n "S{Q[n]} " 
44 
ANS alse “expr $n $ SLINEWIDTH' -eq 0 ] # Format output. 
46 then # ^ Siglo 


47 echo # Break lines into neat chunks. 


48 
49 
50 
51 
52 
33 
54 
55 
56 
57 
58 
59 


fru 


done 


echo 


exit 0 


Se SH ch ok 


3 Syuieb GB SSIS a 


[This is an iterative implementation of the Q-series. 
[The more intuitive recursive implementation is left as an exercise. 


Warning: calculating this series recursively takes a V 
C/C++ would be orders of magnitude faster. 


ERY long time 


Bash supports only one-dimensional arrays, though a little trickery permits simulating multi-dimensional 


ones. 


Example 27-17. Simulating a two-dimensional array, then tilting it 


NPRPRPRPP PPP PY 
O (o 00 -1 O Oi i$ (). M P. O xo 0 - O O1 4 CQ I 


(G9 (e) TROP NSF ee ISS) INO) SS) SP Soy dS) 
I=) c9 Wer (Ger c] (ex, (nl PS. S d» [5 


C0 CO CO CO CO CO CO CO 
wor (Ge) s] dex; Gal UES ey) du» 


op 
i c» 


#!/bin/bash 


# twodim.sh: Simulating a two-dimensional array. 


# A one-dimensional array consists of a single row. 
# A two-dimensional array stores rows sequentially. 


Rows-5 
Columns-5 
# 5 X 5 Array. 


declare -a alpha 


load alpha () 
( 

local rc=0 
local index 


inus Sk shia JN Je: (C 1D) de 


let "index - 


Moo eccL IET 
done 
# Simpler would be 


# char alpha [Rows] [Columns]; 
# Unnecessary declaration. Why? 


pom Gp gUBMNGOREeU RUSO UM OXON 
do # Use different symbols if you like. 
local row- 'expr $rc / $Columns~ 
local column- expr Src $ S$Rows' 
$row * SRows + Scolumn" 
alpha [Sindex]=Si 
# alpha[$row] [$column] 


#+ Cecile =a msi A JE (C. D dm P E 3b a Kh MNO 19 (v R SW p W WP x xt) 
#+ but this somehow lacks the "flavor" of a two-dimensional array. 


} 

Daine seudlime (() 
( 

local row=0 
local index 


echo 


manke [p See" 


ng 


"SRows" ] ao Diino «Owe sum "ees was: oroen 


42 
43 
44 
45 
46 
47 
48 
49 
50 
E 
52 
53 
54 
59 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
pi 
72 
13 
74 
US 
76 
33 
78 
US 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
9i 
92 
DS 
94 
25) 
96 
2 
98 
99 
100 
OL 
102 
103 
104 
105 
106 
107 


do #+ columns vary, 
#+ while row (outer loop) remains the same. 
local column=0 
echo -n " " # Lines up "square" array with rotated one. 
weist be || WSereylivintia' lic - VSXerox brise st | 
do 
let "index = $row * SRows + $column" 
echo -n "S{alpha[index]} " # alone Sron] [$column] 
let e cumsan 
done 
let "row += 1" 
echo 
done 
# The simpler equivalent is 


# echo $(alpha[*]) | xargs -n $Columns 
echo 
} 
FERESTRE) # Filter out negative array indices. 
( 
echo -n " " 44$ Provides the tilt. 
# Explain how. 

i£ [ “SY ege Q0 sc "$1" cit "Sous" me "SZV -oge 0 CE VEZ) cl 
then 

let "index = $1 * SRows + $2" 

# Now, print it rotated. 

echo -a V Salona imez] bY 

# alpha[$row] [$column] 
itu 
} 
rotate () # Rotate the array 45 degrees -- 
{ #+ "balance" it on its lower lefthand corner. 
local row 


local column 


tom (( row = RONS wow = Rowse dgeexy-— J) 
do # Step through the array backwards. Why? 
fors (( colum = 0} column «uobumase Solvin JI 
do 
ar [ Groni -ee O | 
then 
ier Vel = Selon = Sse) 
lee "Eg = Seluma” 
else 
let "tl = $column" 
let "t2 = Scolumn + Srow" 
aft 
Tabee Sieil SicZ # Filter out negative array indices. 
# What happens if you don't do this? 


done 


"SColumns" 


1] 


10S echo; echo 

LLO 

111 done 

112 

113 4$ Array rotation inspired by examples (pp. 143-146) in 

114 #+ "Advanced C Programming on the IBM PC," by Herbert Mayer 
115 #+ (see bibliography). 

116 # This just goes to show that much of what can be done in C 
117 #+ can also be done in shell scripting. 

HERO 

Lis j 

12/0 

dL 21 

122 s Now, let the show begin. # 

123 load_alpha # Load the array. 

124 print_alpha ur erint Le OWE. 

125 rotate # Rotate it 45 degrees counterclockwise. 

126 # # 

TA 

128 eaw © 

128) 

130) This is a rather contrived, not to mention inelegant simulation. 
TS 

132 Exercises: 

133 «p ————————- 

134 1) Rewrite the array loading and printing functions 

133 in a more intuitive and less kludgy fashion. 

136 

137) 2) Figure out how the array rotation functions work. 

11318 Hint: think about the implications of backwards-indexing an array. 
13g 

140 3) Rewrite this script to handle a non-square array, 

141 such as a 6 X 4 one. 

142 Try to minimize "distortion" when the array is rotated. 


A two-dimensional array is essentially equivalent to a one-dimensional one, but with additional addressing 
modes for referencing and manipulating the individual elements by row and column position. 


For an even more elaborate example of simulating a two-dimensional array, see Example A-10. 


For more interesting scripts using arrays, see: 


* Example 12-3 
* Example 16-46 
* Example A-22 
* Example A-44 
* Example A-41 
* Example A-42 
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Chapter 28. Indirect References 


We have seen that referencing a variable, Svar, fetches its value. But, what about the value of a value? What 
about $$var? 


The actual notation is | $5var, usually preceded by an eval (and sometimes an echo). This is called an 
indirect reference. 


Example 28-1. Indirect Variable References 


#!/bin/bash 
# ind-ref.sh: Indirect variable referencing. 
# Accessing the contents of the contents of a variable. 


s; Buceo letts tool. around a lale. 
var=23 


il 

2 

3 

4 

5 

6 

7 

8 

9 echo "\Svar = Svar" # Svar = 23 
10 So far, everything as expected. But 

il 

2 

2 

4 

5 

6 

3) 

8 


Echo UMS WSEHe = povai T SSwar = 45 70weie 
Not useful 
NS \owexpanded to PID of the Seriot 
== reker tO tbe entry om the SS variabla == 
+ and "var" is echoed as plain text. 
(Thank you, Jakob Bohm, for pointing this out.) 


19 echo "\\\$\S$var = N$$var" # \SSvar = $23 


20 As expected. The first $ is escaped and pasted on to 
21 #+ the value of var ($var = 23 ). 

22 Meaningful, but still not useful. 

23 

24 Now, let's start over and do it the right way. 

25 

26 # 

27] 

28 


29 a-letter of alphabet 4 Variable "a" holds the name of another variable. 
30 letter of alphabet-z 


EN 

32 echo 

33 

34 Direct reference. 

35 echo Ya = Sav # a = letter_of_alphabet 

36 

ST Indirect reference. 

38 eval a=\$Sa 

39 OE Forcing an eval(uation), and 

40 2. Escaping the first $ 

41 

42 The 'eval' forces an update of $a, sets it to the updated value of \S$Sa. 
43 So, we see why 'eval' so often shows up in indirect reference notation. 
44 

45 echo "Now a = $a" # Now a = Zz 

46 

47 echo 

48 

49 


50 # Now, let's try changing the second-order referenc 


Syl 
52 
53 
54 
55 
56 
57 
58 
3$ 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
vel 
72 
TES) 
74 
WS 
76 
T3 
78 
WE 


t=table_cell_3 
table cell 3-224 


echo "\"table_cell_3\" = $table cell 3" $ rable gell 39" e 24 
cho -n "dereferenced \"t\" = "; eval echo \$$t $t glewetesemced "p" = 24 
# In this simple case, the following also works (why?). 

# vall qose echo TATEN = Sic 

echo 

t-table cell 3 


NEW VAL-387 
table cell 3-$NEW VAL 
echo "Changing value of NV"table cell 3N" to $NEW VAL." 
acho VwWUEale cell 33V mew Stabla cell 3 
cho =n Ydearefarenced WEN now "n eval echo \SSic 
# "eval" takes the two arguments "echo" and "\$St" (set equal to $table cell 3) 


echo 

# (Thanks, Stephane Chazelas, for clearing up the above behavior.) 

# A more straightforward method is the ${!t} notation, discussed in the 
sdb Usach, version 2 mex 


# See also ex78.sh. 


eae (0) 


Indirect referencing in Bash is a multi-step process. First, take the name of a variable: varname. Then, 
reference it: $varname. Then, reference the reference: $$varname. Then, escape the first $: 
\$Svarname. Finally, force a reevaluation of the expression and assign it: eval newvar=\$$varname. 


Of what practical use is indirect referencing of variables? It gives Bash a little of the functionality of pointers 
in C, for instance, in table lookup. And, it also has some other very interesting applications. . . . 


Nils Radtke shows how to build "dynamic" variable names and evaluate their contents. This can be useful 
when sourcing configuration files. 


#!/bin/bash 


# 
# This could be "sourced" from a separate file. 
isdnMyProviderRemoteNet=172.16.0.100 
isdnYourProviderRemoteNet=10.0.0.10 
isdnOnlineService="MyProvider" 


# 

remoteNet=$ (eval "echo \S$(echo isdn${isdnOnlineService}RemoteNet) ") 
remoteNet=$ (eval "echo \$$ (echo isdnMyProviderRemoteNet)") 
remoteNet=S (eval "echo \S$isdnMyProviderRemoteNet") 

remoteNet=S (eval "echo $isdnMyProviderRemoteNet") 

echo "SremoteNet" 5 LIZ LOO. LOO 

# 


# And, it gets even better. 


22 
25 
24 
25 
26 
2 
28 
219 
30 
Sal 
32 
33 
34 
35 
36 
37 
38 
3g 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sil 
52 


# 
#+ 


ch 


ge 
un 
ch 
ec 


(lai 


Se ob oc 


Consider the following snippet given a variable named getSparc, 
but no such variable getIa64: 
kMirrorArchs () { 
alee ln er urs 
aie [p YSleval Peeing Ye Stacie Gres (Sele, ne Sess | 
sel Uc \ (a Wat Wie || ite Yaad Yan Ie Celie fessi. | 
SEC "gus (os NOI WIE 3) grade )w = sue] 
then 
return 0; 
else 
Cerura iy 
T1 


tSparc-"true" 
set getIa64 


kMirrorArchs sparc 
ho $? # 0 

# True 
kMirrorArchs Ia64 
ho $? # 1 

# False 
NOCES 


Even the to-be-substituted variable name part is built explicitly. 
The parameters to the chkMirrorArchs calls are all lower case. 
The variable name is composed of two parts: "get" and "Sparc" 


Example 28-2. Passing an indirect reference to awk 


NPRPRPPRP PPP PY 
SCOMIDTDHKRWNHHFPOWOAIDRGUBRWNE 


[SOULS TROY TSS) [SSF oR BIS 
Sy] (px Gal dE (63. de» (5 


N 
[99] 


WN 
C We) 


j 


di. 


/bin/bash 


Another version of the "column totaler" script 
that adds up a specified column (of numbers) in the target file. 
This one uses indirect references. 


ARGS-2 
E WRONGARGS-85 


3LiE 


th 


fta 


ica 
co 


[ $4 -ne "SARGS" ] 4 Check for proper number of command-line args. 
en 

echo "Usage: 'basename $0' filename column-number" 

exit $E WRONGARGS 


lename-$1 # Name of file to operate on. 
lumn number-$2 # Which column to total up. 


==== Same as original script, up to this point =====# 


A multi-line awk script is invoked by 
awk " 


Begin awk script. 


31 # 
32 ugs U 

38 

34 { total += N$$(column number) # Indirect reference 
235 } 

SG END) i 

37 prine tored 

38 } 

39 

40 " "$filename" 

41 # Note that awk doesn't need an eval preceding oes 
42 # 
43 # End awk script. 

44 

45 # Indirect variable reference avoids the hassles 

46 #+ of referencing a shell variable within the embedded awk script. 
47 # Thanks, Stephane Chazelas. 

48 

49 

50 exit $? 


This method of indirect referencing is a bit tricky. If the second order variable changes its value, 
then the first order variable must be properly dereferenced (as in the above example). 
Fortunately, the $ { ! variable} notation introduced with version 2 of Bash (see Example 
36-2 and Example A-22) makes indirect referencing more intuitive. 


Bash does not support pointer arithmetic, and this severely limits the usefulness of indirect referencing. In 


fact, indirect referencing in a scripting language is, at best, something of an afterthought. 
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Chapter 29. /dev and /proc 


A Linux or UNIX filesystem typically has the / dev and /proc special-purpose directories. 


29.1. /dev 


The /dev directory contains entries for the physical devices that may or may not be present in the hardware. 
[1] Appropriately enough, these are called device files. As an example, the hard drive partitions containing the 
mounted filesystem(s) have entries in / dev, as df shows. 


bash$ df 
Filesystem JLTs—Iod.exelss Used Available Use$ 
Mounted on 
/ dev/hda6 495876 222748 247527 48% / 
/dev/hdal 507/55 StS 7) 44248 9% /boot 
/ dev/hda8 367013 13262 334803 4% /home 
/dev/hda5 1714416 LIZS 624 503704 70% /usr 


Among other things, the / dev directory contains loopback devices, such as /dev/1loop0. A loopback 
device is a gimmick that allows an ordinary file to be accessed as if it were a block device. [2] This permits 
mounting an entire filesystem within a single large file. See Example 17-8 and Example 17-7. 


A few of the pseudo-devices in / dev have other specialized uses, such as /dev/null, /dev/zero, 
/ dev /urandom, /dev/sda1 (hard drive partition), /dev/udp (User Datagram Packet port), and 
/ dev/tcp. 


For instance: 
To manually mount a USB flash drive, append the following line to /etc/fstab. [3] 


iL dew ele /mnt/flashdrive auto noauto,user,noatime 0 0 


(See also Example A-23.) 


Checking whether a disk is in the CD-burner (soft-linked to / dev / hdc): 


1 head -1 /dev/hdc 

2 

3 

4 head: cannot open '/dev/hdc' for reading: No medium found 

5) (No disc in the drive.) 

6 

7 head: error reading '/dev/hdc': Input/output error 

8 (There is a disk in the drive, but it can't be read; 

9 #+ possibly it's an unrecorded CDR blank.) 
10 
ial Stream of characters and assorted gibberish 
12 (There is a pre-recorded disk in the drive, 
13 #+ and this is raw output -- a stream of ASCII and binary data.) 
14 Here we s the wisdom of using 'head' to limit the output 
15 #+ to manageable proportions, rather than 'cat' or something similar. 
16 
allay 


18 Now, it's just a matter of checking/parsing the output and taking 
19 #+ appropriate action. 


When executing a command ona /dev/tcp/Shost/Sport pseudo-device file, Bash opens a TCP 
connection to the associated socket. 


A socket is a communications node associated with a specific I/O port. (This is analogous to a hardware 
socket, or receptacle, for a connecting cable.) It permits data transfer between hardware devices on the same 


machine, between machines on the same network, between machines across different networks, and, of 
course, between machines at different locations on the Internet. 


The following examples assume an active Internet connection. 


Getting the time from nist.gov: 


bash$ cat «/dev/tcp/time.nist.gov/13 
530652 (94-0 9—165 (4226854 6S 0 (0 502.5. Wile (NST) 9 


[Mark contributed the above example.] 
Downloading a URL: 


bash$ exec 5«»/dev/tcp/www.net.cn/80 
bash$ echo -e "GET / HTTP/1.0\n" >&5 
bash$ cat «&5 


[Thanks, Mark and Mihai Maties.] 


Example 29-1. Using /dev/tcp for troubleshooting 


#!/bin/bash 
# dev-tcp.sh: /dev/tcp redirection to check Internet connection. 


# Script by Troy Engel. 
# Used with permission. 


TCP_HOST=www.dns-diy.com # A known spam-friendly ISP. 
TCP PORT-80 # Port 80 is http. 


# Try to connect. (Somewhat similar to a 'ping' . . .) 
ho "HEAD / HTTP/1.0" »/dev/tcecp/S(TCP-HOST)/S(TCP-PORT) 
MYEXIT-$? 


: <<EXPLANATION 

If bash was compiled with nable-net-redirections, it has the capability of 
using a special character device for both TCP and UDP redirections. These 
redirections are used identically as STDIN/STDOUT/STDERR. The devic ntries 
are 30,36 for /dev/tcp: 


Mop pop opp PPP PY 
O (o 0» -1 O Oi i (). NM P. O xo 0 -1 O O1 4 CQ I EO 
M 
Q 


mknod /dev/tcp c 30 36 


NON 
NO EX 


>From the bash reference: 
/dev/tcp/host/port 

If host is a valid hostname or Internet address, and port is an integer 
port number or service name, Bash attempts to open a TCP connection to the 
corresponding socket. 
EXPLANATION 


NNN 
Ow C) 


IO) 
1 oO 


N 
[99] 


ie (p “RGM! = UOV jy them 
echo "Connection successful. Exit code: $MYEXIT" 
else 
echo "Connection unsuccessful. Exit code: SMYEXIT" 
i£ 


CO CO CO CO CO CO CO Dd 
fon) (Gal OY (69) [ey [= COSCO 


exit SMYEXIT 


Example 29-2. Playing music 


NPRPRPPRP PPP PY 
COMDIDGTHRWNFOCOMIRDUBRWNHE 


NO NO BO PO NO PO P2 
sup cxy (ar dE (se) ISS) qp 


N 
[99] 


CO CO CO CO CO CO CO CO CO CO Dd 
Gey (Ger c] yy (ub fes (el SS) [L5 e» Ke) 


!/bin/bash 
music.sh 


MUSIC WITHOUT EXTERNAL FILES 


Author: Antonio Macchi 
Used in ABS Guide with permission 


/dev/dsp default = 8000 frames per second, 8 bits per frame (1 byte), 
+ 1 channel (mono) 


duration=2000 # If 8000 bytes = 1 second, then 2000 = 1/4 second. 
Volume oon sse) V i Web. Sweden e = yrk (exe New) c 

mute=$'\x80' # No volume = \x80 (the middle). 

function mknote () # $1=Note Hz in bytes (e.g. A = 440Hz 


{ #+ 8000 fps / 440 = 16 :: A = 16 bytes per second) 
for t in seq 0 $duration' 
do 
test S(( Su > Sil )) e  & echo sa Swolune || ecko -m Smor 
done 


n-'mknote 32767' 
# European notation. 


echo =n "SgqSe2sdScsdScsasgqsnsgqsesnegse2Sdscsesbsescissnseissd N 
Sn$g$e2Sd$cS$d$cS$a$gSns$gSeSn$g$a$d$cSbSa$Sb$c" > /dev/dsp 
# dsp = Digital Signal Processor 


40 exit # A "bonny" example of a shell script! 
Notes 
[1] The entries in /dev provide mount points for physical and virtual devices. These entries use very little 


drive space. 


Some devices, such as /dev/null, /dev/zero, and /dev/urandom are virtual. They are not 
actual physical devices and exist only in software. 


A block device reads and/or writes data in chunks, or blocks, in contrast to a character device, which 
acesses data in character units. Examples of block devices are hard drives, CDROM drives, and flash 
drives. Examples of character devices are keyboards, modems, sound cards. 


Of course, the mount point /mnt /flashdrive must exist. If not, then, as root, mkdir 
/mnt/flashdrive. 


To actually mount the drive, use the following command: mount /mnt/flashdrive 


Newer Linux distros automount flash drives in the /media directory without user intervention. 
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29.2. /proc 


The /proc directory is actually a pseudo-filesystem. The files in /proc mirror currently running system and 
kernel processes and contain information and statistics about them. 


bash$ cat /proc/devices 
Character devices: 
1 mem 
pty 
ttyp 
Bays 
cua 
7 vcs 
10 misc 
14 sound 
ZG fb 
36 netlink 
128 ptm 
136 PES 
162 raw 
254 pcmcia 


Ow WN 


Block devices: 
1 ramdisk 

2 wel 

3 ide0 

9 md 


bash$ cat /proc/interrupts 


CPUO 
0 84505 XT-PIC timer 
I: SS 75 XT-PIC keyboard 
DR 0 XT-PIC cascade 
5s ib XT-PIC  soundblaster 
8 il XOP-IUHNO PEE 
128 4231 XT-PIC PS/2 Mouse 
14: 199375 XT-PIC ideO 
NMI: 0 
ERR: 0 


bash$ cat /proc/partitions 


major minor #blocks name rio rmerge rsect ruse wio wmerge wsect wuse running use aveq 
3 0 3007872 hda 4472 22260 114520 94240 3551 18703 50384 549710 0 111550 644030 
3 il 52416 hdal 27 395 844 960 4 2 14 180 0 800 1140 
9 2 lick? 0000000000 0 
3 4 165230 hcas 10 © 20 210 0 0 0 0 (9 210 210 


bash$ cat /proc/loadavg 
(39 de 0,27 2/44 wis) 


bash$ cat /proc/apm 
lale 3.2. (zx S 00L Oxi 0x30 —J5 = 2 


bash$ cat /proc/acpi/battery/BATO/info 


joue exse, § yes 

design capacity: 43200 mWh 
last dull Capacity: 36640 mWh 
battery technology: rechargeable 
design voltage: 10800 mV 
design capacity warning: 1832 mWh 
design capacity low: 200 mWh 


(exea cuecinillaicniny ie il sum 
capacity granularity 2: 1 mWh 


model number: IBM-02K6897 
serial number: Liss 
battery type: LION 

OEM info: Panasonic 


bash$ fgrep Mem /proc/meminfo 
MemTotal: 505218 lg 
MemFree: 266248 kB 


Shell scripts may extract data from certain of the files in /proc. [1] 


1 FS-iso # ISO filesystem support in kernel? 


3 grep SFS /proc/filesystems # iso9660 


1 kernel version-$( awk '( print $3 }' /proc/version ) 


1 CPU=$( awk '/model name/ {print $5}' « /proc/cpuinfo ) 

2 

3 aie | UBXOENUU = USES S | 

4 then 

5 run_some_commands 

6 ws 

7 else 

8 run_other_commands 

9 eae 

MORF 

LL 

12 

13 

l4 cpuemspeed-s(tgrep “cpu MHz" /proc/cpuinto | awk “{fprint S4) j 
15 Current operating speed (in MHz) of the cpu on your machine. 
16 On a laptop this may vary, depending on use of battery 
17 #+ or AC power. 

1 #!/bin/bash 

2 get-commandline.sh 

3 Get the command-line parameters of a process. 

4 

5 OPTION=cmdline 

6 

7 4 Identify PID. 

8 pid=$( echo $(pidof "$1") | awk '{ print $1 }' ) 

9 # Get only first DOS SSS SS OO arn T E SEO PICS Cauet elige sto 
10 
11 echo 
12 echo Process ID OF (HELMS Instanes gi) YALU = Soich 
13 echo -n "Command-line arguments: " 
14 cat /proc/"Spid"/"SOPTION" | xargs -0 echo 
15 # Formats output: (ENSIS KE EE 


16 # (Thanks, Han Holl, for the fixup!) 


JU) 


18 echo; echo 

19 

20 

21 # For example: 


22 # sh get-commandline.sh xterm 
+ 

1 devfile="/proc/bus/usb/devices" 

2 text-"Spd" 

3 USB1="Spd=12" 

4 USB2="Spd=480" 

5 

6 

7 bus speed-$(fgrep -m 1 "$text" Sdevfile | awk '(print $9}") 
8 4 ^^ SEO edtices iuge ub 
E 

10 if [ "$bus speed" - "SUSBI" ] 

11 then 

12 Geno "USE i. joie Toe 

i3 # Do something appropriate for USB 1.1. 

Jd du 


8^; It is even possible to control certain peripherals with commands sent to the /proc directory. 


root# echo on > /proc/acpi/ibm/light 


This turns on the Thinklight in certain models of IBM/Lenovo Thinkpads. (May not work on all Linux 


distros.) 


Of course, caution is advised when writing to /proc. 


The /proc directory contains subdirectories with unusual numerical names. Every one of these names maps 
to the process ID of a currently running process. Within each of these subdirectories, there are a number of 
files that hold useful information about the corresponding process. The stat and status files keep running 
statistics on the process, the cmdline file holds the command-line arguments the process was invoked with, 
and the exe file is a symbolic link to the complete path name of the invoking process. There are a few more 
such files, but these seem to be the most interesting from a scripting standpoint. 


Example 29-3. Finding the process associated with a PID 


! /bin/bash 
pid-identifier.sh: 
Gives complete path name to process associated with pid. 


ARGNO-1 # Number of arguments the script expects. 
WRONGARGS=65 

PID=66 

E_NOSUCHPROCESS=67 

E NOPERMISSION-68 

PROCFILE-exe 


E p pH 
Ww 
D 
©] 


if [ $4 -ne SARGNO ] 

then 
echo "Usage: ~basename $0! PID-number" >&2 # Error message »stderr. 
exit $E WRONGARGS 

ftii 


jostieheve- S ( pe ax || greo Sil || amk "X peime Sil Jj" || pas Sil ) 
up Checka ioe oie aim Wie Iagsiesine, Prelec il. 
# Then makes sure it is the actual process, not the process invoked by this script. 


21 
22 
29 
24 
25 
26 
ZI 
28 
219 
20 
S 
32 
39 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sil 
52 
53 
54 
95 
56 
5 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
qi 
U2 
vis 
74 
1S 
76 
77 
78 
79 
80 
81 
82 
83 
84 
85 


The last "grep $1" filters out this possibility. 


# 
# 
# pidno=$( ps ax eue "X arime Sil X || erreg Sl) 
# also works, as Teemu Huovila, points out. 
nie || =z VSioacine'! ] If, after all the filtering, the result is a zero-length string, 
then + no running process corresponds to the pid given. 

echo "No such process running." 

exit SE_NOSUCHPROCESS 


Alternatively: 
mie | js Sil = /clew/mull 25d 
then no running process corresponds to the pid given. 
echo "No such process running." 
exit SE NOSUCHPROCESS 
Fi 


To Ssumpelstiy Cia ntire process, use "pidof". 


me [ | s Vpro SL /SEPROCE TUEN | # Check for read permission. 


Scho Widiocess Sil ibiousaliove,, lowie . co 
echo "Can't get read permission on /proc/$1/$PROCFILE." 
exit $E NOPERMISSION # Ordinary user can't access some files in /proc. 


# The last two tests may be replaced by: 
d ai | kiril —0 Si = /scew/fmullil 2l s: VO" as mor a Sicjinel, lowe 


# this will test whether it is possible 
# to send a signal to the process. 

# then echo "PID doesn't exist or you're not its owner" >&2 

# exit $E BADPID 

# fa 

Aa eiS ( iss Sl Zexsexe/ dL || reges Vesa wwe Yi raine isalal gu 5 

# Or ME eile=S( is = /oroc/Sil/exe || eis T fjeienimie Salil}? }) 

# 

#  /proc/pid-number/exe is a symbolic link 


#+ to the complete path name of the invoking process. 


if [ -e "Sexe file" ] # If /proc/pid-number/exe exists, 

then #+ then the corresponding process exists. 
echo "Process #$1 invoked by $exe file." 

else 
echo "No such process running." 

fai 


This elaborate script can *almost* be replaced by 
joe: cbx || xguesyo Sul [| we Vit Tewesbewe uS gU 
However, this will not work... 
* because the fifth field of 'ps' is argv[0] of the process, 
ar aor elm xecutable file path. 


However, either of the following would work. 
minds Aore ol/exes prints! s\n" 
legt -arm -9 Sil =C we | Secl -mne "ms ^. p" 


Additional commentary by Stephane Chazelas. 


e» (0) 


Example 29-4. On-line connect status 


Mop pop pp pppÀ: 
O (o 0» -1 O Oi i (). M PB. O xo 0 -1 O O1 4 CQ Io E 


[NX DSF o» TSS) SS) (Sy de» 
=} (m. (ap des (9 IS) [5 


GE 00 C0 C0 C0 WWW CO CO CO h2 DN 
O We) Ce) b oy (Onde Cy I) | & We) (ee) 


oe 
um 


QO fF SF PP SP uS aum aum 
(&» (xe) (Ce SS) (ox Gi de (3. [3 


(On nh (On Cah Gab vei 
(ox (On. dex ©) e» 1 


!/bin/bash 


PROCNAME-pppd # ppp daemon 
PROCFILENAME-status 4 Where to look. 
NOTCONNECTED=65 

INTERVAL=2 # Update every 2 seconds. 


Lu 


pidno=$( ps ax | grep -v "ps ax" | grep -v grep | grep SPROCNAM 
exu "Jp joucakione Sil gv p 


Finding the process number of 'pppd', the 'ppp daemon'. 
Have to filter out the process lines generated by the search itself. 


However, as Oleg Philon points out, 
* this could have been considerably simplified by using "pidof". 
pidno-$( pidof SPROCNAME ) 


Moral of the story: 
+ When a command sequence gets too complex, look for a shortcut. 


af | —z V"Spaeimeo" | # If no pid, then process is not running. 
then 

echo "Not connected." 

exit SNOTCONNECTED 
else 

echo "Connected."; echo 


FL 
while [ true ] # Endless loop, script can be improved here. 
do 

if [ ! -e "/proc/Spidno/SPROCFILENAME" ] 

# While process running, then "status" file exists. 

then 


echo "Disconnected." 


exit SNOTCONNECTED 

ita 
netstat -s | grep "packets received" # Get some connect statistics. 
netstat -s | grep "packets delivered" 

Sleep SINTERVAL 

echo; echo 
done 
exit 0 
# As it stands, this script must be terminated with a Control-C. 
# Exercises: 
$ ER mmm m m 
# Improve the script so it exits on a "q" keystroke. 
# ake the script more user-friendly in other ways. 


In general, it is dangerous to write to the files in /proc, as this can corrupt the filesystem or crash the 
machine. 


Notes 


[1] Certain system commands, such as procinfo, free, vmstat, Isdev, and uptime do this as well. 
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Chapter 30. Of Zeros and Nulls 


Faultily faultless, icily regular, splendidly null 
Dead perfection; no more. 
--Alfred Lord Tennyson 

/dev/zero... /dev/null 


Uses of /dev/null 
Think of /dev/nu11 as a black hole. It is essentially the equivalent of a write-only file. Everything 
written to it disappears. Attempts to read or output from it result in nothing. All the same, 
/ dev/null can be quite useful from both the command-line and in scripts. 


Suppressing stdout. 


1 cat $filename >/dev/null 
2 ae Coments Cie che Pile mill poe list ico Sicckowic . 


Suppressing stderr (from Example 16-3). 


1 rm Sbadname 2>/dev/null 
2 t So error messages [stderr] deep-sixed. 


Suppressing output from both stdout and stderr. 


cat $filename 2>/dev/null »/dev/null 

If "Sfilename" does not exist, there will be no error message output. 

If "Sfilename" does exist, the contents of the file will not list to stdout. 
Therefore, no output at all will result from the above line of code. 


This can be useful in situations where the return code from a command 


Jd 
2 
3 
4 
5 
6 
7 #+ needs to be tested, but no output is desired. 
8 


E cat $filename &>/dev/null 
10 also works, as Baris Cicek points out. 


Deleting contents of a file, but preserving the file itself, with all attendant permissions (from Example 
2-1 and Example 2-3): 


1 cat /dev/null > /var/log/messages 
2 # : > /var/log/messages has same effect, but does not spawn a new process. 
3 


4 cat /dev/null » /var/log/wtmp 
Automatically emptying the contents of a logfile (especially good for dealing with those nasty 
"cookies" sent by commercial Web sites): 


Example 30-1. Hiding the cookie jar 


# Obsolete Netscape browser. 
# Same principle applies to newer browsers. 


l 

2 

3 

4 if [ -f ~/.netscape/cookies ] # Remove, if exists. 
5 then 
6 
7 
8 
9 


rm -f ~/.netscape/cookies 
iE aL 


In -s /dev/null ~/.netscape/cookies 


10 


# All cookies now get sent to a black hole, rather than saved to disk. 


Uses of / dev/zero 


Like /dev/null, /dev/zero is a pseudo-device file, but it actually produces a stream of nulls 
(binary zeros, not the ASCII kind). Output written to /dev/zero disappears, and it is fairly difficult 
to actually read the nulls emitted there, though it can be done with od or a hex editor. The chief use of 
/ dev/zerois creating an initialized dummy file of predetermined length intended as a temporary 


swap file. 


Example 30-2. Setting up a swapfile using /dev/zero 


NPRPRPPRP PPP PY 
O io 0» -1 O Oi i$ (0. NM IB. O xo 0 -1 O) O1 i CQ) I E 


(RO) TSS) [sey (SS) [soy oy [se 
=) cem, (rl gEy (99 [sy [3 


N 
© 


CO CO CO CO CO CO CO CO CO CO Dd 
We) (e c op) (Da ges (63) he [S (9) (we 


OP fF Pb Pe PP SP a4 
(S) We) (eo) | fp) (ar des te) [n5 £3 S&S 


[91 
ES 


!/bin/bash 
Creating a swap file. 


A swap file provides a temporary storage cache 
+ which helps speed up certain filesystem operations. 


ROOT UID-0 # Root has SUID O0. 
E_WRONG_USER=85 # Not root? 


FILE=/swap 
BLOCKSIZE=1024 
MINBLOCKS=40 
SUCCESS=0 


# This script must be run as root. 

x I VUSIUJIID" -me VSIROOW GODU T 

then 
echo; echo "You must be root to run this script."; echo 
exit $E WRONG USER 

ie aL 


blocks-$(1:-$MINBLOCKS] # Set to default of 40 blocks, 


#+ if nothing specified on command-line. 
This is the equivalent of the command block below. 
ae [| =a VSL ] 
then 
blocks=$1 
else 
blocks-$MINBLOCKS 
E 
aie | “VSloloeks” -lr ENINELOCKS J 
then 
blocks=$MINBLOCKS # Must be at least 40 blocks long. 
fal 


FE HE TE FE TE FE HE FE FE FE HE TE FE HE FE HE FE E FE EEE HEHEHE EH HEHEHE FE FE HE FE HE EE HEE EEE EEE EE HEHEHE EE 
echo "Creating swap file of size $blocks blocks (KB)." 
dd if=/dev/zero of-S$FILE bs=SBLOCKSIZE count-$blocks # Zero out file. 


mkswap SFILE Sblocks Designate it a swap file. 
swapon SFILE Activate swap file. 
retcode=$? Everything worked? 


# Note that if one or more of these commands fails, 
#+ then it could cause nasty problems. 
HEHEHE EEE E E E E E E E EEE EERE EERE E ERE RE ERE ERE ERE E EE E EE ERE EEE HEH HE HHH HHH EHF 


59 
60 
61 
62 
63 


# Exercise: 

# Rewrite the above block of code so that if it does not execut 
#+ successfully, then: 

d 1) an error message is echoed to stderr, 

# 2) all temporary files are cleaned up, and 

# 3) the script exits in an orderly fashion with an 

#+ appropriate error code. 

echo "Swap file created and activated." 


exit Sretcode 


Another application of /dev/zero is to "zero out" a file of a designated size for a special purpose, 
such as mounting a filesystem on a loopback device (see Example 17-8) or "securely" deleting a file 


(see Example 16-60). 


Example 30-3. Creating a ramdisk 


nm 


PRP PRR 
O0 -1 OY O1 4 C). 9. EB. O (0 O0 -1 O OI dS (Q I9 ES 


NO NO) NO PS r 
(eS) [ry f O Wo) 


NNN NY 
=| ey (on oS 


N 
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!/bin/bash 
ramdisk.sh 


A "ramdisk" is a segment of system RAM memory 
+ which acts as if it were a filesystem. 
Its advantage is very fast access (read/write time). 
Disadvantages: volatility, loss of data on reboot or powerdown. 
T less RAM available to system. 


Of what use is a ramdisk? 
Keeping a large dataset, such as a table or dictionary on ramdisk, 
+ speeds up data lookup, since memory access is much faster than disk access. 


E NON ROOT USER-70 # Must run as root. 
ROOTUSER NAME-root 


OUNTPT-/mnt/ramdisk 

SIZE-2000 # 2K blocks (change as appropriate) 
BLOCKSIZE-1024 # 1K (1024 byte) block size 
DEVICE-/dev/ram0 # First ram device 


username= id -nu' 
if [ "$username" != "SROOTUSER NAME" ] 
then 

echo "Must be root to run \"` basename $0^N"." 
exit $E NON ROOT USER 


ial 
di [ | e SOUNS | Test whether mount point already there, 
then T SO) io errom ILE tuls SCL as deum 
mkdir SMOUNTPT + multiple times. 
ial 
Ha HE EEE E TE EE HE TE FE HE EEE FE HE EH FE HE ERE RE EE EE HE EE ERE EE E E E EE EEE 


dd if=/dev/zero of-$DEVICE count-$SIZE bs=SBLOCKSIZE # Zero out RAM device. 
# Why is this necessary? 


mke2fs SDEVICE Create an ext2 filesystem on it. 

mount SDEVICE SMOUNTPT Movie LEs 

chmod 777 SMOUNTPT Enables ordinary user to access ramdisk. 
However, must be root to unmount it. 


FEFE HE EE HE TE EE E TE HE EEE FE HE EEE FE E EE EE EE EE HE HE E EEE EE HE ERE ERE EE HE HE E E E E EH E E EE EE 
# Need to test whether above commands succeed. Could cause problems otherwis 
# Exercise: modify this script to make it safer. 


45 
46 echo "\"SMOUNTPT\" now available for use." 


47 The ramdisk is now accessible for storing files, even by an ordinary user. 
48 

49 Caution, the ramdisk is volatile, and its contents will disappear 

50 #+ on reboot or power loss. 

Bil Copy anything you want saved to a regular directory. 

52 

53 After reboot, run this script to again set up ramdisk. 

54 Remounting /mnt/ramdisk without the other steps will not work. 

55 

56 Suitably mocaikiecd, chais seripe Cain by invoked in J/GXECU IG soU re- local, 
57 #+ to set up ramdisk automatically at bootup. 

58 That may be appropriate on, for example, a database server. 

59 

60 exit 0 


In addition to all the above, /dev/ zero is needed by ELF (Executable and Linking Format) 
UNIX/Linux binaries. 
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Chapter 31. Debugging 


Debugging is twice as hard as writing the code in 
the first place. Therefore, if you write the code as 
cleverly as possible, you are, by definition, not 
smart enough to debug it. 


--Brian Kernighan 
The Bash shell contains no built-in debugger, and only bare-bones debugging-specific commands and 
constructs. Syntax errors or outright typos in the script generate cryptic error messages that are often of no 
help in debugging a non-functional script. 


Example 31-1. A buggy script 


1 #!/bin/bash 

2 # ex74.sh 

3 

4 # This is a buggy script. 
5 4$ Where, oh where is the error? 
6 

7 a=37 

8 

9 si [Sm ge 27 | 

10 then 

Ail echo $a 

12 Gab 

13 

l4 exit 0 


Output from script: 


./ex74.sh: [37: command not found 


What's wrong with the above script? Hint: after the if. 


Example 31-2. Missing keyword 


#!/bin/bash 
# missing-keyword.sh: What error message will this generate? 


T 

2 

3 

4 iow m im 1 2 3 
3 Co 

6 echo "Sa" 

7 
8 
9 


# done # Required keyword 'done' commented out in line 7. 


e» d) 


Output from script: 
missing-keyword.sh: line 10: syntax error: unexpected end of file 


Note that the error message does not necessarily reference the line in which the error occurs, but the line 
where the Bash interpreter finally becomes aware of the error. 


Error messages may disregard comment lines in a script when reporting the line number of a syntax error. 


What if the script executes, but does not work as expected? This is the all too familiar logic error. 


Example 31-3. test24: another buggy script 


1 #!/bin/bash 

2 

3 # This script is supposed to delete all filenames in current directory 
4 #+ containing embedded spaces. 
5 # It doesn't work. 

6 # Why not? 

7 

8 

9 badname-' 1s | grep ' "^ 
10 
dil e "nx ENTS; 
12 # echo "$badname" 
13 
14 rm "$badname" 
15 
16 exit 0 


Try to find out what's wrong with Example 31-3 by uncommenting the echo "$badname" line. Echo 
statements are useful for seeing whether what you expect is actually what you get. 


In this particular case, rm "S$badname" will not give the desired results because $badname should not be 
quoted. Placing it in quotes ensures that rm has only one argument (it will match only one filename). A partial 
fix is to remove to quotes from $badname and to reset $IFS to contain only a newline, IFS=$'\n'. 
However, there are simpler ways of going about it. 


1 # Correct methods of deleting filenames containing spaces. 
2 wm WW F 

3 rm *" "x 

Al seg 7o Ws 

5 # Thank you. S.C. 


Summarizing the symptoms of a buggy script, 
1. It bombs with a "syntax error" message, or 
2. It runs, but does not work as expected (logic error). 
3. It runs, works as expected, but has nasty side effects (logic bomb). 


Tools for debugging non-working scripts include 


1. Inserting echo statements at critical points in the script to trace the variables, and otherwise give a 
snapshot of what is going on. 


į ) Even better is an echo that echoes only when debug is on. 


### debecho (debug-echo), by Stefano Falsetto ### 
### Will echo passed parameters only if DEBUG is set to a value. ### 
debecho () { 
if [ ! -z "SDEBUG" ]; then 
exo VENU $2 
# Sino s te clei: 


(Sy ike) feo Sa] tony (al ses (O3) deor I 


DEBUG=on 


11 Whatever-whatnot 

12 debecho SWhatever # whatnot 

13 

14 DEBUG= 

15 Whatever=notwhat 

16 debecho $Whatever # (Will not echo.) 


2. Using the tee filter to check processes or data flows at critical points. 
3. Setting option flags -n -v -x 


sh -n scriptname checks for syntax errors without actually running the script. This is the 
equivalent of inserting set -n or set -o noexec into the script. Note that certain types of 
syntax errors can slip past this check. 


sh -v scriptname echoes each command before executing it. This is the equivalent of inserting 
set -vorset -o verbose in the script. 


The -n and -v flags work well together. sh -nv scriptname gives a verbose syntax check. 


sh -x scriptname echoes the result each command, but in an abbreviated manner. This is the 
equivalent of inserting set -x or set -o xtrace in the script. 


Inserting set -u or set -o nounset in the script runs it, but gives an unbound variable error 
message at each attempt to use an undeclared variable. 

4. Using an "assert" function to test a variable or condition at critical points in a script. (This is an idea 
borrowed from C.) 


Example 31-4. Testing a condition with an assert 


1 #!/bin/bash 
2 4 assert.sh 


3 
4 EEEE TETE FE HE HE TE FE FE HE TE FE FE FE E TE FE FE HE E TE EE ERE ERE 
5 assert () # If condition false, 
G 4 #+ exit from script 
7 #+ with appropriat rror message. 
8 E_PARAM_ERR=98 
9 E ASSERT FAILED-99 
10 
L 
12 mip oq m “EQN ] # Not enough parameters passed 
13 then #+ to assert() function. 
14 return $E PARAM ERR # No damage done. 
L5 Ea 
16 
17 lineno=$2 
18 
19 sae [4 eb 
20 then 
2T edho Usi: flee Siyon 
22 echo "File N"SON", line $lineno" # Give name of file and line number. 
2:5 exit $E ASSERT FAILED 
24 # else 
25 # return 
26 # and continue executing the script. 
2 ipa 
28 ) $ Insert a similar assert() function into a script you need to debug. 


29 FEAE AE HEHE IE AE HHT HE HHT HH EH FE AE FE HE AE FE FE FE FE EE ae Ea EEE aE 
30 


St 
32 
33 
34 
35 
36 
37 
38 
3 
40 
41 
42 
43 
44 
45 
46 
47 
48 


a=5 
b=4 
condition="Sa le Sb" 


# 
# 
#+ 


assert "Scondition" SLINENO 


Some commands. 
Some more commands 


More commands 


exit $? 


Error message and exit from script. 
Try setting "condition" to something else 
and see what happens. 


The remainder of the script executes only if the "assert" does not fail. 


echo "This statement echoes only if the \"assert\" does not fail." 


5. Using the $LINENO variable and the caller builtin. 
6. Trapping at exit. 


The exit command in a script triggers a signal 0, terminating the process, that is, the script itself. [1] It 
is often useful to trap the exit, forcing a "printout" of variables, for example. The trap must be the first 
command in the script. 


Trapping signals 


trap 


Specifies an action on receipt of a signal; also useful for debugging. 


A signal is a message sent to a process, either by the kernel or another process, telling it to take 
some specified action (usually to terminate). For example, hitting a Control-C sends a user interrupt, 
an INT signal, to a running program. 


A simple instance: 


il 
2 
3 
4 
5 


Trap YY 2 
# Ignore interrupt 2 


(Control-C), with no action specified. 


tre "exe MComerol—c chiseled." 2 
# Message when Control-C pressed. 


Example 31-5. Trapping at exit 


(sey =a] fey) (ab des te) (3) [5 


Ko) 


dal 
12 


!/bin/bash 
Hunting variables with a trap. 


trap 'echo Variable Listing --- a = 
EXIT is the name of the signal generated upon exit from a script. 


Sa Io e SS" TaT 


The command specified by the "trap" doesn't execute until 
+ the appropriate signal is sent. 


echo "This prints before the \"trap\" --" 
echo "even though the script sees the \"trap\" first." 


echo 


43 

14 a=39 

15 

16 b-36 

aly 

18 exit 0 

19 # Note that commenting out the 'exit' command makes no difference, 
20 #+ since the script exits in any case after running out of commands. 


Example 31-6. Cleaning up after Control-C 


#!/bin/bash 
# logon.sh: A quick 'n dirty script to check whether you are on-line yet. 


umask 177 # Make sure temp files are not world readable. 


,OGFILE=/var/log/messages 

ote that SLOGFILE must be readable 

+ (as root, chmod 644 /var/log/messages). 

PFILE-temp.$$ 

Create a "unique" temp file name, using process id of the script. 
Using 'mktemp' is an alternative. 

For example: 

TEMPFILE-'mktemp temp.XXXXXX' 

KEYWORD-address 
At logon, the line "remote IP address xxx.xxx.xxx.xxx" 
appended to /var/log/messages. 


Mop op op ppppmpogÀ: 
O (o 0» -1 O Oi i (). M PB. O xo 0 -1 O O1 4 CQ I9 E 
rH 
E 
S 


O 

USER_INTERRUPT=13 
CHECK LINES-100 
# 


21l 
22 How many lines in log file to check. 
23 
24 trap 'rm -f STEMPFILE; exit S$USER INTERRUPT' TERM INT 
25 4$ Cleans up the temp file if script interrupted by control-c. 
26 
27 echo 
28 

while [ STRUE ] #Endless loop. 

do 

tail —-n SCHECK_ LINES SLOGFILE> $TEMPETILDLE 


# Saves last 100 lines of system log file as temp fil 
# Necessary, since newer kernels generate many log messages at log on. 
search=grep SKEYWORD $TEMPFILE' 
# Checks for presence of the "IP address" phrase, 
#+ indicating a successful logon. 


CO CO CO CO CO CO CO CO CO CO Dd 
We) ee} SE tony (Ont. des Gd) I) dE (e) Ko) 


ase | 9 —- WS | Quotes necessary because of possible spaces. 
then 

40 echo "On-line" 

41 rm -f STEMPFILE Clean up temp file. 

42 exit SONLINE 

43 else 

44 echon =i YY The -n option to echo suppresses newline, 

45 + so you get continuous rows of dots. 

46 ak 

47 

48 sleep 1 

49 done 

50 


Bl 


52 # Note: if you change the KEYWORD variable to "Exit", 

53 #+ this script can be used while on-lin 

54 #+ to check for an unexpected logoff. 

55 

56 # Exercise: Change the script, per the above note, 

57 d cuo JOSIE IL IEW LEs 

58 

5:9) esate. 10) 

60 

61 

62 # Nick Drage suggests an alternate method: 

63 

64 while true 

65 do ifconfig pppd | grep UP 1> /dev/null && echo "connected" && exit 0 
66 echo n "." PAPEN ES AAOS (Lom go ) until connected. 

67 sleep 2 

68 done 

69 

70 # Problem: Hitting Control-C to terminate this process may be insufficient. 
7l #+ (Dots may keep on echoing.) 

72 4 Exercise: Fix this. 

TS 

74 

75 

76 # Stephane Chazelas has yet another alternativ 

73 

78 CHECK INTERVAL-1 

p 

80 while ! tail -n 1 "SEOGEILE"U | grep -q "SKEYWORD" 

81 do echo -n 

82 sleep $CHECK INTERVAL 

83 done 

84 echo "On-line" 

85 
86 # Exercise: Discuss the relative strengths and weaknesses 
87 # of each of these various approaches. 


°) The DEBUG argument to trap causes a specified action to execute after every command in a script. This 
permits tracing variables, for example. 


Example 31-7. Tracing a variable 


#!/bin/bash 


trap 'echo "VARIABLE-TRACE» \S$variable = \"Svariable\""' DEBUG 
# Echoes the value of Svariable after every command. 


variable-29 


echo " Just initialized \$variable to $variable." 


let "variable *= 3" 


exelo V geer unbllienjoilayeyel Wehweuesnelo ks lox 3, 
exit 
s^ eve Urre "euowamsucoll a s o CWiimsincl a a o” IDRNBUU(G exowsEicwuiel as 


#+ more appropriate in the context of a complex script, 
#+ where inserting multiple "echo $variable" statements might be 
#+ awkward and time-consuming. 


PRPPPPRPRPRPHE EB 
XO 00 1 OY O1 4 (0 I9 L2. O o 0 1 O) O1 4 QI) ES 


20 4 Thanks, Stephane Chazelas for the pointer. 


N 
H 


22 


23 (üxucjguhE OQ SewUjdIE s 

24 

25 VARIABLE-TRACE» Svariable = "" 

26 VARIABLE-TRACE> Svariable = "29" 

2 duse anitaalized Svariable to 29° 
28 VARIABLE-TRACE» Svariable = "29" 

29 VARIABLE-TRACE» Svariable = "87" 

30 Just multiplied $variable by 3. 
31 VARIABLE-TRACE» $variable = "87" 


Of course, the trap command has other uses aside from debugging, such as disabling certain keystrokes 
within a script (see Example A-43). 


Example 31-8. Running multiple processes (on an SMP box) 


NPRPPPRP PPP PY 
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#!/bin/bash 

# parent.sh 

# Running multiple processes on an SMP box. 
# Author: Tedman Eng 


G6 WMS ie the ficer OI WIS Ses. 
#+ both of which must be present in the current working directory. 


LIMIT=$1 # Total number of process to start 
NUMPROC=4 # Number of concurrent threads (forks?) 
PROCID=1 # Starting Process ID 


echo "My PID is $$" 


function start thread() { 
if [ $PROCID le SLIMIT | ; then 
./child.sh $PROCIDé 
let "PROCID++" 


else 
echo "Limit reached." 
wait 
SALE 
TI 
} 
vaile Jp “ENUMEROCH =e Q ip cho 
start thread, 
let "NUMPROC--" 
done 
while true 
do 
trap "start thread" SIGRTMIN 
done 
e» O 
# ======== Second script follows ======== 


48 #!/bin/bash 
ANS) s lor Ile) Sila 


50 # Running multiple processes on an SMP box. 
51 # This script is called by parent.sh. 

52 # Author: Tedman Eng 

53 

54 temp=$RANDOM 

55 index=$1 

56 misce 


Gy kee Vice; = SY 

58 let "temp += 4" 

59 cho WSieeicieiine, Since. Timed Sicene” VS 

60 sleep $(temp) 

61 echo "Ending Sindex" 

62 ka s STGREIMENCSPPETD 

63 

64 exit O0 

65 

66 

67 SCRIPT AUTHOR'S NOTI 
68 It's not completely bug free. 

69 I ran it with limit - 500 and after the first few hundred iterations, 
70 #+ one of the concurrent threads disappeared! 

Tal Not sure if this is collisions from trap signals or something else. 

72 Once the trap is received, there's a brief moment while executing the 
73 #+ trap handler but before the next trap is set. During this time, it may 
74 #+ be possible to miss a trap signal, thus miss spawning a child process. 
15 
76 No doubt someone may spot the bug and will be writing 

UU xe. n on x dügt eE TOE WAS, 

78 

78 

80 

81 4 # 
82 

83 

84 

85 # # 
86 

87 

88 

89 FHEEEEE HE EE EEE EE HEH HE HE EEE EE EH HE EE EEE EE HEH HE EE ERE EE HEH HE EE HE EH 

90 The following is the original script written by Vernia Damiano. 

CRI Unfortunately, it doesn't work properly. 

92 FERETE AE AE EEE HE HE EEE HE HEE HEHE HE EEE EE E E E E E E E E E E HEH EE EE EE E E EH HE HE HE EE 

93 
94 #!/bin/bash 


[Ea] 
ca 
E 


95) 
96 Must call script with at least one integer parameter 
97 #+ (number of concurrent processes). 
98 All other parameters are passed through to the processes started. 
99 
100 
101 INDICE-8 # Total number of process to start 
102 TEMPO-5 # Maximum sleep time per process 
103 E BADARGS-65 # No arg(s) passed to script. 
104 


105 if [ S4 -eq 0 ] 4 Check for at least one argument passed to script. 
106 then 


ILO echo "Usage: `basename $0` number_of_processes [passed params]" 
108 exit $E BADARGS 

LOS) 3651 

110 

111 NUMPROC-$1 # Number of concurrent process 

WAZ Blase 


113 PARAMETRI-( "S@" ) # Parameters of each process 


i14 
US) fenetiomn avvia 4 


116 local temp 

LL local index 

118 temp=SRANDOM 

1LJL S) index-$1 

120 emat Iie 

121 let "temp %= STEMPO" 
122 let "temp += 1" 

123 echo "Starting $index Time:$temp" "S@" 
124 sleep S{temp} 

125 echo "Ending Sindex" 
126 kill -s SIGRTMIN $$ 
WF 

128 

129) itwumci3uem perci) 4 

130 ase [ STDICE gu © ] p chen 
i avvia SINDICE "S{PARAMETRI[@]}" & 
11512 let WATINID ITC 
133 else 

134 trap : SIGRTMIN 
135 Fai 

136 j 

LEF 

138 trap parti SIGRTMIN 

13S 

140 while [ "SNUMPROC" -gt 0 ]; do 
141 parti; 

142 let "NUMPROC--" 

143 done 

144 

145 wait 

146 trap - SIGRTMIN 

147 

Me uetus E 

149 


150 : ««SCRIPT AUTHOR COMMENTS 
151 I had the need to run a program, with specified options, on a number of 
152 different files, using a SMP machine. So I thought [I'd] keep running 


153 a specified number of processes and start a new one each time . . . one 
154 of these terminates. 
155 


156 The "wait" instruction does not help, since it waits for a given process 
157 or *all* process started in background. So I wrote [this] bash script 
158 wee cam clo ile Jao, Geine tine Vicia” igmetiUIebigims 

115) --Vernia Damiano 

160 SCRIPT AUTHOR COMMENTS 


£j trap '' SIGNAL (two adjacent apostrophes) disables SIGNAL for the remainder of the script. trap 
SIGNAL restores the functioning of SIGNAL once more. This is useful to protect a critical portion of a 
script from an undesirable interrupt. 


il ieee UV 2 a Sicgmel 2 is Comerol—C, mow clisaloilecl. 
2 command 

3 command 

4 command 

5 wrea 2 # Reenables Control-C 

6 


Version 3 of Bash adds the following internal variables for use by the debugger. 


1. SBASH_ARGC 


Number of command-line arguments passed to script, similar to 
2. SBASH, ARGV 


Final command-line parameter passed to script, equivalent S$ { ! 
3. SBASH, COMMAND 


Command currently executing. 
4. SBASH, EXECUTION STRING 


The option string following the —c option to Bash. 
5. SBASH, LINENO 


In a function, indicates the line number of the function call. 
6. SBASH REMATCH 


Array variable associated with =~ conditional regex matching. 
7. SBASH_SOURCE 


Same as $0. 
8. SBASH SUBSHELL 


Notes 


[1] By convention, signal 0 is assigned to exit. 
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Chapter 32. Options 


Options are settings that change shell and/or script behavior. 


The set command enables options within a script. At the point in the script where you want the options to take 
effect, use set -o option-name or, in short form, set -option-abbrev. These two forms are equivalent. 


1 !/bin/bash 

2 

3 set -o verbose 

4 Echoes all commands before executing. 
5 

1 !/bin/bash 

2 

3 set -v 

4 Exact same effect as above. 

E 


# To disable an option within a script, use set +o option-name or set +option-abbrev. 


#!/bin/bash 


set -o verbose 
# Command echoing on. 
command 


command 
set +o verbose 
# Command echoing off. 


command 
# Not echoed. 


NPRPPRPRP PPP PY 
O (o 0» 1 O Ui i (). M. I. O (o 0 -1 O OI i CQ) Io e 


Set X 
# Command echoing on. 
command 
command 
Al set +v 
27 # Command echoing off. 
23 command 
24 
25 exit 0 
26 


An alternate method of enabling options in a script is to specify them immediately following the 7 ! script 
header. 


#!/bin/bash -x 
T 
# Body of script follows. 


fs (d$ [m» [25 


It is also possible to enable script options from the command line. Some options that will not work with set 
are available this way. Among these are —i, force script to run interactive. 


bash -v script-name 


bash -o verbose script-name 


The following is a listing of some useful options. They may be specified in either abbreviated form (preceded 
by a single dash) or by complete name (preceded by a double dash or by —-o). 


Table 32-1. Bash options 


Abbreviation 


-B Enable brace expansion (default setting = on) 


+B brace Disable brace expansion 
expansion 


Prevent overwriting of files by redirection (may be overridden by >l) 


pe LE List double-quoted strings prefixed by $, but do not execute commands in 
script 


-a  lalexpot | Export all defined variables 


WEE AE Notify when jobs running in background terminate (not of much use in a 
script) 


iI, Informs user of any open jobs upon shell exit. Introduced in version 4 of 

Bash, and still "experimental." Usage: shopt -s checkjobs (Caution: may 

hang!) 

EE EE Abort script at first error, when a command exits with non-zero status 
(except in until or while loops, if-tests, list constructs) 

=£ þoglob_  [Fienameexpanson(glbbing disabled | 


Enables the ** globbing operator (version 4+ of Bash). Usage: shopt -s 
globstar 


gl 
Script runs in interactive mode 
-n 


Read commands in script, but do not execute them (syntax check) 


Invoke the Option-Name option 


Attempt to use undefined variable outputs error message, and forces an 
exit 


Unset positional parameters. If arguments given (-- argl arg2), 
positional parameters set to arguments. 
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Chapter 33. Gotchas 


Turandot: Gli enigmi sono tre, la morte una! 
Caleph: No, no! Gli enigmi sono tre, una la vita! 


--Puccini 


Here are some (non-recommended!) scripting practices that will bring excitement into an otherwise dull life. 


Assigning reserved words or characters to variable names. 


(8 c (xy (Ga des (3) [3X p 


«o 


4.2 


case-valueO # Causes problems. 
23skidoo-valuel # Also problems. 


Variable names starting with a digit are reserved by the shell. 
Try | 23skidoo-valuel. Starting variables with an underscore is okay. 


However . . . using just an underscore will not work. 

225 

echo $ . # S is a special variable set to last arg of last command. 
IURE. a n 6 .. is a valid function name! 

xyz((!*-value2 # Causes severe problems. 


As of version 3 of Bash, periods are not allowed within variable names. 


* Using a hyphen or other reserved characters in a variable name (or function name). 


i 
2 
3 
4 
5 
6 
7 
8 


9 
10 


var-1=23 

USE Vwarus dL" suene 
function-whatever () # Error 

Use 'function whatever ()' instead. 

As of version 3 of Bash, periods are not allowed within function names. 
function.whatever () # Error 

Use 'functionWhatever ()' instead. 


* Using the same name for a variable and a function. This can make a script difficult to understand. 


(8) 3) cx) (rl des Ge) Jm» I 


«o 


LO 


do_something () 


{ 


echo "This function does something with \"S$1\"." 


do something-do something 


do something do something 


# 


ALL chwis als legal, louie Iniciniiy comrems ing . 


* Using whitespace inappropriately. In contrast to other programming languages, Bash can be quite 
finicky about whitespace. 


(S5) =] fy) (Gy dEs te) [X [5 


varl = 23 # 'varl-23' is correct. 

# On line above, Bash attempts to execute command "vari" 

# with the arguments "=" and "23". 

lec gG = Sa, SIS # Instead: let c-$a-$b or let Ye = $a = Sig 
if [ Sia -le 5] # if [ $a -le 5 ] is correct. 

# oe a [ “Sal -la 5 ] is even better. 


9 * LL $a -le 5 1] also works. 
e 


Not terminating with a semicolon the final command in a code block within curly brackets. 


{ iig Sip cles echo "leues }} 
# bash: syntax error: unexpected end of file 


v ds aie “dis echo “Dona, Yo 4 
# 5 ### Final command needs semicolon. 


(Om: dE (8) SS) p 


Assuming uninitialized variables (variables before a value is assigned to them) are "zeroed out". An 
uninitialized variable has a value of null, not zero. 


1 #!/bin/bash 

2 

3 echo "uninitialized var = $uninitialized var" 
4 4$ uninitialized var = 


Mixing up = and -eq in a test. Remember, = is for comparing literal variables and -eq for integers. 


Taf ee Sia = 273 l # Is $a an integer or string? 
2 as | VUSa" -e0 279 ! # If $a is an integer. 

3 

4 # Sometimes you can interchang q and = without adverse consequences. 
5 # However 

6 

7 

8 a=273.0 # Not an integer. 

9 

IO a2 [| "Saws = 273 | 

11 then 

12 echo "Comparison works." 

13 else 

14 echo "Comparison does not work." 

J5 Ea # Comparison does not work. 

16 

17 # Same with a=! 273" ancl mU ys. 

18 

1$ 
20 4 Likewise, problems trying to use "-eq" with non-integer values. 
Adl 
22 mi | Spat -e0 279.0 | 
23 then 
24 echo "a = $a" 
25 fi # Aborts with an error message. 
26 4 test.sh: [: 273.0: integer expression expected 


Misusing string comparison operators. 


Example 33-1. Numerical and string comparison are not equivalent 


#!/bin/bash 
# bad-op.sh: Trying to use a string comparison on integers. 


il 

2 

3 

4 echo 

5 number=1 
6 

7 # The following while-loop has two errors: 
8 #+ one blatant, and the other subtle. 

9 


10 while [ "Snumber" « 5 ] # Wrong! Should be: while [ "Snumber" -1t 5 ] 


i lo 

1,2 echo -n "$number " 
L3 let "number += 1" 
14 done 


15 # Attempt to run this bombs with th rror messag 

16 #+ bad-op.sh: line 10: 5: No such file or directory 

17 # Within single brackets, "<" must be escaped, 

18 #+ and even then, it's still wrong for comparing integers. 


19 

20 echo " w 

2L 

22 while [ "$number" \< 5 ] u dh o3 Sud 

23 do # 

24 echo -n "Snumber " # It *seems* to work, but 

25 let "number += 1" #+ it actually does an ASCII comparison, 
26 done #+ rather than a numerical one. 
27 

28 echo; cho M n 

29 

30 # This can cause problems. For example: 

Sil 


32 lesser=5 
33 greater=105 


34 

35 ic | "SgweeEec" \< "Silessee” j 

36 then 

37 echo "Sgreater is less than $lesser" 

JS t3 # 105 is less than 5 


39 s^ Jum tact, YlOS” actually ig less itum YEW 
40 #+ in a string comparison (ASCII sort order). 


42 echo 


44 exit 0 


Attempting to use let to set string variables. 


1 let "a - hello, you" 
2l echo Usan # 0 


LJ 
Sometimes variables within "test" brackets ([ ]) need to be quoted (double quotes). Failure to do so 
may cause unexpected behavior. See Example 7-6, Example 20-5, and Example 9-6. 

e 


Quoting a variable containing whitespace prevents splitting. Sometimes this produces unintended 
consequences. 


Commands issued from a script may fail to execute because the script owner lacks execute permission 
for them. If a user cannot invoke a command from the command-line, then putting it into a script will 
likewise fail. Try changing the attributes of the command in question, perhaps even setting the suid bit 
(as root, of course). 

e. 
Attempting to use - as a redirection operator (which it is not) will usually result in an unpleasant 
surprise. 


commandl 2» - | command2 
# Trying to redirect error output of commandl into a pipe 
u^ e o n WELLE snore work: 


[Og dE (69 ISS) J 


commandl 2>& - | command2 # Also futile. 


6 
7 


Tioanks SEa 


Using Bash version 2+ functionality may cause a bailout with error messages. Older Linux machines 
may have version 1.XX of Bash as the default installation. 


[b 
O Wey (e| xp ny Gal des Ge [SS fe 


fies 
A 


PRR 
DB WN 


LS 


!/bin/bash 
minimum_version=2 
Since Chet Ramey is constantly adding features to Bash, 
you may set $minimum version to 2.XX, 3.XX, or whatever is appropriate. 
E BAD VERSION-80 
LE | WSBVASIE VRSTO A< U'Ssnababu dL vers omt ] 
then 
echo "This script works only with Bash, version Sminimum or greater." 
echo "Upgrade strongly recommended." 
exit $E BAD VERSION 
ie al 


* Using Bash-specific functionality in a Bourne shell script (#! /bin/sh) on a non-Linux machine 
may cause unexpected behavior. A Linux system usually aliases sh to bash, but this does not 
necessarily hold true for a generic UNIX machine. 


Using undocumented features in Bash turns out to be a dangerous practice. In previous releases of this 
book there were several scripts that depended on the "feature" that, although the maximum value of 
an exit or return value was 255, that limit did not apply to negative integers. Unfortunately, in version 
2.05b and later, that loophole disappeared. See Example 24-9. 


A script with DOS-type newlines ( Vr V2) will fail to execute, since #! /bin/bash\r\n is not 
recognized, not the same as the expected #! /bin/bash\n. The fix is to convert the script to 


UNIX-style newlines. 


ies 
© Wey Qo sa) rex Gal des Ge [eS P 


[Es 
[zn 


PRR 
Dm Wh 


#!/bin/bash 


echo "Here" 


unix2dos $0 


chmod 755 $0 


./80 


echo "There" 


exit 0 


# Script changes itself to DOS format. 
# Change back to execute permission. 
# The 'unix2dos' command removes execute permission. 


t oeripe teries to run hee again. 
# But it won't work as a DOS file. 


A shell script headed by #! /bin/sh will not run in full Bash-compatibility mode. Some 
Bash-specific functions might be disabled. Scripts that need complete access to all the Bash-specific 
extensions should start with #! /bàn/bash. 
e Putting whitespace in front of the terminating limit string of a here document will cause unexpected 
behavior in a script. 
* Putting more than one echo statement in a function whose output is captured. 


de G9 m» pL 


add2 () 
{ 


echo! “Whatever rt # Delete this line! 


let "retval 


TE 


5 echo Sretval 

6 } 

7 

8 num1-12 

9 num2=43 
LO echo "Sum of Snuml and Snum2 = $(add2 $numl $num2)" 
qu 
12 # Sum of 12 and 43 = Whatever 
13 # 55 
14 
15 # The "echoes" concatenate. 


This will not work. 

e. 
A script may not export variables back to its parent process, the shell, or to the environment. Just as 
we learned in biology, a child process can inherit from a parent, but not vice versa. 


1 WHATEVER-/home/bozo 
2 export WHATEVER 
3 emu ( 


bash$ echo S$WHATEVER 


bash$ 
Sure enough, back at the command prompt, $WHATEVER remains unset. 
e. 
Setting and manipulating variables in a subshell, then attempting to use those same variables outside 
the scope of the subshell will result an unpleasant surprise. 


Example 33-2. Subshell Pitfalls 


1 #!/bin/bash 

2 4 Pitfalls of variables in a subshell. 

3 

4 outer variable-outer 

5 echo 

6 echo "outer variable = $outer variable" 

7 echo 

8 

9 ( 

10 # Begin subshell 

alt 

12 echo "outer_variable inside subshell = Souter_variable" 
13 inner_variable=inner # Set 

14 echo "inner variable inside subshell = Sinner_variable" 
15 outer variable-inner # Will value change globally? 

16 echo "outer variable inside subshell = $outer variable" 
17 

18 # Will 'exporting' make a difference? 

19 # export inner_variable 
20 # export outer_variable 
21 # Try it and see. 
22 
23 # End subshell 
24 ) 
25 
26 echo 
AY «cimo "ime variable Ovicsuicl stubehslil = Sinner variablat + Uh. 
28 echo "outer variable outside subshell = $outer variable" # Unchanged. 
29 echo 
30 
31 exit 0 


[99 
N 


33 4 What happens if you uncomment lines 19 and 20? 
34 4 Does it make a difference? 


Piping echo output to a read may produce unexpected results. In this scenario, the read acts as if it 
were running in a subshell. Instead, use the set command (as in Example 15-18). 


Example 33-3. Piping the output of echo to a read 


#!/bin/bash 

# badread.sh: 

# Attempting to use 'echo and 'read' 

#+ to assign variables non-interactively. 


a-aaa 
b-bbb 
E=ECE 


echo "one two three" | readab c 
" duy BS KSASSGiN Bp, lo, aincl Gu 


echo 

echo "a = $a" # a = aaa 
echo "b = $b" # b = bbb 
echo e Sc tac — coe 


# Reassignment failed. 


# 


NPRPRPRPRP PPP PY 
O io 00 -1 O Oi i& (0. NN HL O «o 0 -1 O OI iS CQ IO 


21 4$ Try the following alternative. 
22 
23 var= echo "one two three" 
24 set -- $var 
25 mesi: b=S2p ese 
26 
Z7 ximo. d —————-—— " 
28 echo "a = $a" # a - one 
echo "b = $b" # b = two 
echo "c = $c" # c = three 


# Reassignment succeeded. 


# Note also that an echo to a 'read' works within a subshell. 
# However, the value of the variable changes *only* within the subshell. 


a=aaa # Starting all over again. 


CO CO CO CO CO CO CO CO CO CO Dd 
We) ge) -— (ex; Gr ges (63) hy [S (9) ve 


echo; echo 


40 
4 
42 
43 echo "one two three" | ( read a b c; 
44 eoo Vimsidoe suoshells Ys echo Us = Sa" echo Yo = $o; echo Ye = Sev j 
45 # a = one 
46 # b = two 
47 # c = three 
48 echo " N 
49 echo "Outside subshell: " 
50 echo "a = $a" 4 a = aaa 
echo "b = Sb" # b = bbb 
echo "c = $c" d c = ccc 


aoa uo 
ES) 1S) 1S 
[0] 
Q 
= 
[9] 


Du cutie (0) 


In fact, as Anthony Richardson points out, piping to any loop can cause a similar problem. 


1 # Loop piping troubles. 

2 # This example by Anthony Richardson, 

3 #+ with addendum by Wilbert Berendsen. 

4 

5 

6 foundone-false 

7 find S$HOME -type f -atime +30 -size 100k | 

8 while true 

9) clo 

10 read f 

it echo "Sf is over 100KB and has not been accessed in over 30 days" 
2 echo "Consider moving the file to archives." 
3 foundone-true 

4 # 

5 echo "Subshell level = SBASH SUBSHELL" 

6 # Subshell level = 1 

7 # Yes, we're inside a subshell. 

8 # 

19 done 


21 # foundone will always be false here since it is 
22 #+ set to true inside a subshell 

23 if [ Sfoundone = false ] 

24 then 

25 echo "No files need archiving." 

PHO ie ab 


28 # Now, here is the correct way: 


30 foundone=false 
31 for f in $(find SHOME -type f -atime +30 -size 100k) # No pipe here. 
32 do 


313 echo "Sf is over 100KB and has not been accessed in over 30 days" 
34 echo "Consider moving the file to archives." 

315 foundone-true 

36 done 

37 

38 if [ Sfoundone = false ] 

39 then 

40 echo "No files need archiving." 

AL 3p 

42 

43 d And here is another alternativ 
44 


45 4$ Places the part of the script that reads the variables 
46 #+ within a code block, so they share the same subshell. 
47 # Thank you, W.B. 


48 

49 find SHOME -type f -atime +30 -size 100k | { 

50 foundone-false 

5l while read f 

52 do 

59 echo "$f is over 100KB and has not been accessed in over 30 days" 
54 echo "Consider moving the file to archives." 
55 foundone=true 

56 done 

57 

58 if ! $foundone 

59 then 


60 echo "No files need archiving." 


61 ít 
62 | 


A lookalike problem occurs when trying to write the stdout of a tail -f piped to grep. 


tail -f /var/log/messages | grep "SERROR_MSG" >> error.log 

# The "error.log" file will not have anything written to it. 
# As Samuli Kaipiainen points out, this results from grep 
#+ buffering its output. 

# The fix is to add the "--line-buffered" parameter to grep. 


Gus Go IND ES 


Using "suid" commands within scripts is risky, as it may compromise system security. [1] 
e. 


Using shell scripts for CGI programming may be problematic. Shell script variables are not 
"typesafe," and this can cause undesirable behavior as far as CGI is concerned. Moreover, it is 
difficult to "cracker-proof" shell scripts. 

* Bash does not handle the double slash (//) string correctly. 


e 

Bash scripts written for Linux or BSD systems may need fixups to run on a commercial UNIX (or 
Apple OSX) machine. Such scripts often employ the GNU set of commands and filters, which have 
greater functionality than their generic UNIX counterparts. This is particularly true of such text 
processing utilites as tr. 

Danger is near thee -- 

Beware, beware, beware, beware. 

Many brave hearts are asleep in the deep. 

So beware -- 


Beware. 


--A.J. Lamb and H.W. Petrie 
Notes 


[1] Setting the suid permission on the script itself has no effect in Linux and most other UNIX flavors. 
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Chapter 34. Scripting With Style 


Get into the habit of writing shell scripts in a structured and systematic manner. Even on-the-fly and "written 
on the back of an envelope" scripts will benefit if you take a few minutes to plan and organize your thoughts 
before sitting down and coding. 


Herewith are a few stylistic guidelines. This is not (necessarily) intended as an Official Shell Scripting 
Stylesheet. 


34.1. Unofficial Shell Scripting Stylesheet 


* Comment your code. This makes it easier for others to understand (and appreciate), and easier for you 
to maintain. 


1 PASS-"SPASSS(MATRIX:$ ((SRANDOM$S$ { #MATRIX})):1}" 

2 # It made perfect sense when you wrote it last year, 
3 #+ but now it's a complete mystery. 

4 d (From Antek Sawicki's "pw.sh" script.) 


Add descriptive headers to your scripts and functions. 


1 #!/bin/bash 

2 

3 ck ck ck ck ck Ck ock ck cock ck ck ck ck ck ck ck ck ck ock ko ckock ck ok ck kock ko ck ck kock ck ck ck ko ck ko ko ck ko Sk ko ko koX 

4 xyz.sh 

5 written by Bozo Bozeman 

6 duly OF, 2001 

7 

8 Clean up project files. 

9 kkkxkxkxkxkxkxkxkxkxkkkkxkxkxkkkkxkxkkkkkxkxkxkkkkxkxkxkxkxkkkxkxkkkkkxkxx*k 

10 

11 E_BADDIR=85 # No such directory. 

12 projectdir-/home/bozo/projects # Directory to clean up. 

13 

14 # 
15 cleanup pfiles () # 
TES Removes all files in designated directory. # 
17 Parameter: S$target directory # 
JL) Returns: 0 on success, $E BADDIR if something went wrong. 4 
1$ # 
20 cleanup_pfiles () 
2 s 
22 se || | ed Wisi q 4p Nes ib tacet Telex Gb) 
29 then 
24 echo “Si sig moc a Ou:ectosy." 
25 return $E BADDIR 
26 iE a 
AT 
28 xp aE WS es 
29 return 0 # Success. 
30 j 
31 
32 cleanup pfiles $projectdir 
33 
34 exit S? 


e Avoid using "magic numbers," [1] that is, "hard-wired" literal constants. Use meaningful variable 
names instead. This makes the script easier to understand and permits making changes and updates 
without breaking the application. 


if [ -f /var/log/messages ] 
then 


iE 3L 

# A year later, you decide to change the script to check /var/log/syslog. 
# It is now necessary to manually change the script, instance by instance, 
#+ and hope nothing breaks. 


# A better way: 

LOGFILE=/var/log/messages # Only line that needs to be changed. 
ie | -2 VGrOCP uA | 

then 


PRR 
WNHROCOAIADABWNHE 


ia fi 
* Choose descriptive names for variables and functions. 


1 fl= 1s -al $dirname' He VOCs 

2 file_listing= ls -al S$dirname' # Better. 

3 

4 

5 MAXVAL-10 # All caps used for a script constant. 

6 while [ "Sindex" -le "SMAXVAL" ] 

7 

8 

9 

10 E NOTFOUND-95 # Uppercase for an errorcode, 
L1 #+ and name prefixed with E_ 
12 s || 4 ee Veille ] 

13 then 

14 echo "File $filename not found." 

LS exit SE_NOTFOUND 

UG su 

L7 

18 

19 MAIL DIRECTORY-/var/spool/mail/bozo # Uppercase for an environmental 
20 export MAIL DIRECTORY #+ variable. 
ait 
22 
23 GetAnswer () # Mixed case works well for a 
Z4 4 #+ function name, especially 
2:5) prompt-$1 #+ when it improves legibility. 
26 echo -n $prompt 
27 read answer 
28 return Sanswer 
29 Jp 
30 
31 GetAnswer "What is your favorite number? " 
32 favorite number-$? 
33 echo $favorite number 
34 
39 
36 _uservariable=23 # Permissible, but not recommended. 


37 # It's better for user-defined variables not to start with an underscore. 
38 # Leave that for system variables. 


* Use exit codes in a systematic and meaningful way. 


1 E WRONG ARGS-95 

2 

EE 

4 exit $E WRONG ARGS 
See also Appendix D. 


Ender suggests using the exit codes in /usr/include/sysexits.h in shell scripts, though these 
are primarily intended for C and C++ programming. 
* Use standardized parameter flags for script invocation. Ender proposes the following set of flags. 


1 -a All: Return all information (including hidden file info). 

2 l9 Brier: Short version, usually tom ouner scis 

3 =@ Copy, concatenate, etc. 

4 -d Daily: Use information from the whole day, and not merely 

5 information for a specific instance/user. 

6 -e Extended/Elaborate: (often does not include hidden file info). 
7 -h Help: Verbose usage w/descs, aux info, discussion, help. 

8 See also -V. 

$9 =l Log GQUESWE (ui SEWE c 

LO =a Manual: Launch man-page for base command. 


LL =m Numbers: Numerical data only. 


12 -r Recursive: All files in a directory (and/or all sub-dirs). 
I3 =e Setup & File Maintenance: Config files for this script. 

14 -u Usages MUSE OF iümvOCcoticm legs tor tle Scip. 

1s cw Verbose: Human readable output, more or less formatted. 

16 -V Version / License / Copy(right|left) / Contribs (email too). 


See also Section F.1. 
* Break complex scripts into simpler modules. Use functions where appropriate. See Example 36-4. 
* Don't use a complex construct where a simpler one will do. 


COMMAND 
ad [| SP eee @ | 


# Redundant and non-intuitive. 


if COMMAND 


(69) cp cx Wal des Ge RS) I 


# More concise (if perhaps not quite as legible). 


... reading the UNIX source code to the Bourne 
shell (/bin/sh). I was shocked at how much simple 
algorithms could be made cryptic, and therefore 
useless, by a poor choice of code style. I asked 
myself, "Could someone be proud of this code?" 


--Landon Noll 
Notes 


[1] In this context, "magic numbers" have an entirely different meaning than the magic numbers used to 
designate file types. 
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Chapter 35. Miscellany 


Nobody really knows what the Bourne shell's 
grammar is. Even examination of the source code 
is little help. 


--Tom Duff 


35.1. Interactive and non-interactive shells and 
scripts 


An interactive shell reads commands from user input on a tt y. Among other things, such a shell reads startup 
files on activation, displays a prompt, and enables job control by default. The user can interact with the shell. 


A shell running a script is always a non-interactive shell. All the same, the script can still access its t t y. It is 
even possible to emulate an interactive shell in a script. 


1 #!/bin/bash 

2 MY PROMPT-'$ ' 
3 while 

4 do 

5 eee =m WSIMA Pome! 
6 read line 

7 eval velina” 
8 done 

9 

10) cxie O 

dal 


12 # This example script, and much of the above explanation supplied by 
13 # Stéphane Chazelas (thanks again). 


Let us consider an interactive script to be one that requires input from the user, usually with read statements 
(see Example 15-3). "Real life" is actually a bit messier than that. For now, assume an interactive script is 
bound to a tty, a script that a user has invoked from the console or an xterm. 


Init and startup scripts are necessarily non-interactive, since they must run without human intervention. Many 
administrative and system maintenance scripts are likewise non-interactive. Unvarying repetitive tasks cry out 
for automation by non-interactive scripts. 


Non-interactive scripts can run in the background, but interactive ones hang, waiting for input that never 
comes. Handle that difficulty by having an expect script or embedded here document feed input to an 
interactive script running as a background job. In the simplest case, redirect a file to supply input to a read 
statement (read variable <file). These particular workarounds make possible general purpose scripts that run 
in either interactive or non-interactive modes. 


If a script needs to test whether it is running in an interactive shell, it is simply a matter of finding whether the 
prompt variable, $PS1 is set. (If the user is being prompted for input, then the script needs to display a 


prompt.) 


su | —m SPSL ] s xo jore@more E 
then 
# non-interactive 


else 
# interactive 


G ara 
Alternatively, the script can test for the presence of option "i" in the $- flag. 


il Case $- am 

Pe cess) # interactive shell 

SEI 

4 *) # non-interactive shell 

9 55 

© a (Courtesy (gue Vin ACO," 3590979) 


However, John Lange describes an alternative method, using the -t test operator. 


1 # Test for a terminal! 

2 

3 fd=0 # stdin 

4 

5 # As we recall, the -t test option checks whether the stdin, [ -t 0 ], 
© jar Oi sSiccloue, | -i d ||, Lin A epi script 1S camine aim a terminal, 
"| ste [| Se USE T 

8 then 

9 echo interactive 

10 else 

11 echo non-interactive 

Oi peal, 

3 

4 

5 # But, as John points out: 

6 # ae | E © | Woes os, woen you're ogee! ain dhexewubisy 

17 # but fails when you invoke the command remotely via ssh. 
18 # So for a true test you also have to test for a socket. 
19 
AQ aie [L =e "ezdi |l 45 Jewish li 
21 then 
22 echo interactive 
23 else 
24 echo non-interactive 
25 3t 


8^) Scripts may be forced to run in interactive mode with the -i option or with a #!/bin/bash —i header. 
Be aware that this can cause erratic script behavior or show error messages even when no error is 
present. 
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35.2. Shell Wrappers 


A wrapper is a shell script that embeds a system command or utility, that accepts and passes a set of 
parameters to that command. [1] Wrapping a script around a complex command-line simplifies invoking it. 
This is expecially useful with sed and awk. 


A sed or awk script would normally be invoked from the command-line by a sed -e 'commands ' or 
awk 'commands '. Embedding such a script in a Bash script permits calling it more simply, and makes it 
reusable. This also enables combining the functionality of sed and awk, for example piping the output of a set 
of sed commands to awk. As a saved executable file, you can then repeatedly invoke it in its original form or 
modified, without the inconvenience of retyping it on the command-line. 


Example 35-1. shell wrapper 


!/bin/bash 


This simple script removes blank lines from a file. 
No argument checking. 


You might wish to add something like: 


E_NOARGS=85 

ae [| em VS |] 

then 

echo "Usage: ^basename SO target-file" 
exit $E NOARGS 

PL 


sed e / S das 


NPRPRPPRP PPP PY 
SCOMIKDTBRWNHFPOWCMIDGURWNE 


Same as 
sed -e '/^$/d' filename 

invoked from the command-line. 
Bil 
2/2 The '-e' means an "editing" command follows (optional here). 
23 VAY sme cha lorxejauewscuew] Gur Line, "SV ime Sneh 
24 This matches lines with nothing between the beginning and the end -- 
25 #+ blank lines. 
26 The 'd' is the delete command. 
29) 
28 Quoting the command-line arg permits 
29 #+ whitespace and special characters in the filename. 
30 
li Note that this script doesn't actually change the target fil 
52 If you need to do that, redirect its output. 
33 
S exit 


Example 35-2. A slightly more complex shell wrapper 


#!/bin/bash 


# subst.sh: a script that substitutes one pattern for 
#+ another in a file, 
#+ i.e., "sh subst.sh Smith Jones letter.txt". 


(Om. gm 9 [R3 [25 


6 Jones replaces Smith. 

7 

8 ARGS=3 # Script requires 3 arguments. 

9 E BADARGS-85 # Wrong number of arguments passed to script. 
10 

JL ase [| Sys x VSARNESY ] 

12 then 

13 echo "Usage: 'basename $0' old-pattern new-pattern filename" 
14 exit $E BADARGS 

WS) 631 

16 

17 old_pattern=$1 

18 new_pattern=$2 

19 

AQ stie [| m eS 7 

21 then 

22 file name-$3 

23 else 

24 echo "File N"$3N" does not exist." 

25 exit $E_BADARGS 

29. E 

27 

28 

29 

30 Here is where the heavy work gets done. 

31 sed -e "s/$old pattern/$new pattern/g" $file name 

32 === 

33 

34 /!GV ds. Xue Coles, ils: Kosgei tte Comaneci alin, S0; 

35 #+ and /pattern/ invokes address matching. 

36 The 'g,' or global flag causes substitution for EVERY 
37 #+ occurence of $old pattern on each line, not just the first. 
38 Read the 'sed' docs for an in-depth explanation. 

3g 


40 exit $? # Redirect the output of this script to write to a file. 


Example 35-3. A generic shell wrapper that writes to a logfile 


!/bin/bash 
Generic shell wrapper that performs an operation 
4- enoi logus nes 


ust set the following two variables. 
OPERATION- 


Can be a complex chain of commands, 
for example an awk script or a pipe 


gi 


„OGF ILI 


Command-line arguments, if any, for the operation. 


OPTIONS="$@" 


(On| WES (U3y IS) [= (» Wey (99) <a) (ox (xb de Ger ISS) [ 
n 


16 4 WOE ae. 

17 echo "date + ^whoami' + SOPERATION "S@"" >> SLOGFIL 
IL Gi; INO, Clo sic. 

19 exec SOPERATION "S@" 

20 

21 # It's necessary to do the logging before the operation. 
22 # Why? 


se | 


Example 35-4. A shell wrapper around an awk script 


!/bin/bash 
pr-ascii.sh: Prints a table of ASCII characters. 


START=33 # Range of printable ASCII characters (decimal). 


il 
2 
3 
4 
5 END=127 # Will not work for unprintable characters (> 127). 
6 
7 echo " Decimal Hex Character" # Header. 
8 echo " i 
9 
10 for ((i=START; i<=END; i++)) 
al clo 
12 Seko Sal | ew Vorname e Sel $2x Quo wm. Bil. Sal, 
13 # The Bash printf builtin will not work in this context: 
14 # jorealinicie Vee eU 
15 done 
16 
Ly «eng f 
18 
19 
20 Decimal Hex Character 
21. = 
27 BS Zale ! 
2 3 34 27 W 
24 35 D d 
25 36 24 $ 
26 
27 
28 
29 1122 7a Z 
30 123 Tho { 
Sd 124 gite: | 
92 dL AS 7d } 
33 
34 
35 Redirect the output of this script to a file 
SS ur (ehe olos le ie "wore B Soi je- aces Sm || esee 


Example 35-5. A shell wrapper around another awk script 


echo "Usage: 'basename $0' filename column-number" 
exit $E WRONGARGS 
iti 


filename-$1 
column number-$2 


(o9] b fon, (mb des Gy) PS) [2 © Wo) (eer cb (ex; (ap odes Go [3 [3 


$1)]' 


! /bin/bash 
Adds up a specified column (of numbers) in the target file. 
Floating-point (decimal) numbers okay, because awk can handle them. 
ARGS-2 
E WRONGARGS-85 
if [ $4 -ne "SARGS" ] 4 Check for proper number of command-line args. 
then 


# Passing shell variables to the awk part of the script is a bit tricky. 


HS One method is to strong-quote the Bash-script variable 
20 #+ within the awk script. 

Dik $'SBASH SCRIPT VAR' 

22 As a 

23 Tula 18 come aim Elm mbedded awk script below. 

24 See the awk documentation for more details. 

25 
26 A multi-line awk script is here invoked by 
AT awk ' 

28 
29 
30 T. 
SL ! 
32 
33 
34 Begin awk script. 
35 
36 awk ' 

37 

33 { coral t= SUVS. muosir] V 
39 f 

40 END { 

41 prine toral 

42 } 

43 

4d " WSR eme 

45 
46 End awk script. 
47 
48 
49 It may not be safe to pass shell variables to an embedded awk script, 
50 #+ so Stephane Chazelas proposes the following alternative: 

Sl 
52 awk -v column number-"$column number" ' 
S { total += $column_number 

54 } 

55 END { 

56 prime CoCa 

57 p" Wishes Leman” 

58 
39 
60 
61 exit 0 


For those scripts needing a single do-it-all tool, a Swiss army knife, there is Perl. Perl combines the 
capabilities of sed and awk, and throws in a large subset of C, to boot. It is modular and contains support for 
everything ranging from object-oriented programming up to and including the kitchen sink. Short Perl scripts 
lend themselves to embedding within shell scripts, and there may be some substance to the claim that Perl can 
totally replace shell scripting (though the author of the ABS Guide remains skeptical). 


Example 35-6. Perl embedded in a Bash script 


#!/bin/bash 


il 
2 
3 # Shell commands may precede the Perl script. 

4 echo "This precedes th mbedded Perl script within N"$0N"." 

5 echo " x 
6 

3i 

8 

E 


perl =f prine “Mais is aim ciuoeleeg! Perl seii. yap" 
# Like sed, Perl also uses the "-e" option. 


IO exeo 7 a 
11 echo "However, the script may also contain shell and system commands." 
12 

13 exit 


It is even possible to combine a Bash script and Perl script within the same file. Depending on how the script 
is invoked, either the Bash part or the Perl part will execute. 


Example 35-7. Bash and Perl scripts combined 


1 #!/bin/bash 

2 bashandperl.sh 

3 

4| dele) Yersetinges rom che Basn parc (out the Gierealjore, SO, 
5 ore Bash commands may follow here. 

6 

7 exit 

8 End of Bash part of the script. 

9 
10 

JLTE 

12 #!/usr/bin/perl 

4.3 This part of the script must be invoked with 

14 perl -x bashandperl.sh 

15 

16 print "Greetings from the Perl part of the script, $0.\n"; 
Jy Perl doesn't seem to like "echo" 

18 ore Perl commands may follow here. 

LE 
20 End of Perl part of the script. 


bash$ bash bashandperl.sh 
Greetings from the Bash part of the script. 


bash$ perl -x bashandperl.sh 
Greetings from the Perl part of the script. 


One interesting example of a complex shell wrapper is Martin Matusiak's undvd script, which provides an 
easy-to-use command-line interface to the complex mencoder utility. Another example is Itzchak Rehberg's 
Ext3Undel, a set of scripts to recover deleted file on an ext3 filesystem. 


Notes 


[1] Quite a number of Linux utilities are, in fact, shell wrappers. Some examples are /usr/bin/pdf2ps, 
/usr/bin/batch,and /usr/bin/xmkmf. 
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35.3. Tests and Comparisons: Alternatives 


For tests, the [[ ]] construct may be more appropriate than [ ]. Likewise, arithmetic comparisons might 
benefit from the (()) construct. 


w 
Il 
œ 


# All of the comparisons below ar quivalent. 

test "Sav -lt 16 te acho “Wes, Sa < 16 rand xin 

Monm/ test Vau ilt 1G Ss echo "wes, Se < io" 

| "Sa? -lt 16 | ee echo “was, Sa < 16” 

[[ $a -lt 16 ]] && echo "yes, $a < 16" # Quoting variables within 

(( a < 16 )) && eco "Wes, Sa < I$" # [[ ]] and (( )) not necessary. 


# Again, all of the comparisons below are equivalent. 

test “Seucy! \< Paris && echo Vios, Paris is greater chan Seaiew 

# Greater ASCII order. 

/bin/test "Scity" \< Paris && echo "Yes, Paris is greater than $city" 
| USgulg/U \< Paris | ce Scho WYyes, Perie le greater them SesieyY 

I erity < Paris |l G& echo Yeo, Paris 1S guste: chan Suy" 

# Need not quote $city. 


i 
2 
3 
4 
5 
6 
Ji 
8 
9 
10 city="New York" 
il 
2 
3 
4 
E 
6 
Ji 
8 
9 


# Thank you, S.C. 
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35.4. A script calling itself (recursion) 


Can a script recursively call itself? Indeed. 


Example 35-8. A (useless) script that recursively calls itself 


!/bin/bash 
recurse.sh 


Can a script recursively call itself? 
Wes, oue LS tils (oue cues; practical mse? 
(See the following.) 


RANGE=10 
[AXVAL-9 
i=SRANDOM 
let "i %= SRANGE" # Generate a random number between 0 and SRANGE - 1. 
ic [ "Si" -le VONVA? | 
then 

echo Mal = Sal 

5d SU) # Script recursively spawns a new instance of itself. 
dE aL # Each child script does the same, until 

#+ a generated $i equals SMAXVAL. 


Mop pop pppppDGÀ:! 
O io 00 -1 O Oi i& (0. M. I. O xo 0 -1 O) O1 i& CQ) I ES 


21 # Using a "while" loop instead of an "if/then" test causes problems. 
22 # Explain why. 

29 

24 exit 0 

25 

26 # Note 

Al xp ===> 

28 # This script must have execute permission for it to work properly. 
29 # This is the case even if it is invoked by an "sh" command. 

30 # Explain why. 


Example 35-9. A (useful) script that recursively calls itself 


#!/bin/bash 
# pb.sh: phone book 


# Written by Rick Boivie, and used with permission. 
# Modifications by ABS Guide author. 


INARGS-1 # Script needs at least one argument. 
DATAFILE-./phonebook 

# A data file in current working directory 
#+ named "phonebook" must exist. 


PROGNAME-$0 


E NOARGS-70 # No arguments error. 


if [ $4 -l1t SMINARGS ]; then 
echo "Usage: "SPROGNAME" data-to-look-up" 
exit $E NOARGS 


(se; c fy) Gl ms ©) Sy) fe) Wer (e) c (ex) (Dat dE (63) SX» [3 


if [ $4 -eq SMINARGS ]; then 


grep $1 "SDATAFILE" 

# 'grep' prints an error message if SDATAFILE not present. 
else 

( xmimsLie Ep "Ue Se y) || efaejs Sil 

# Script recursively calls itself. 
ial 
exit 0 # Script exits here. 

# Therefore, it's o.k. to put 
#+ non-hashmarked comments and data after this point. 

# 
Sample "phonebook" datafile: 
John Doe 1555; Wigan Stay Jessie, MID 21226 (410) 222-3333 
ary Moe 9899 Jones Blvd., Warren, NH 03787 $603) 8985-3232 
Richard Roe SHO jm. Tea Sito, New work, NY 10009 (212) S39 457 
Sam Roe S56 ma ein So; New York, NX DOQUS (ALZ 244-5675 
Zoe Zenobia aM Nia Bye«es Stan Sela, mrare nsen Sir QUSS (4355) 5031 —1 631 
$bash pb.sh Roe 
Richard Roe S56 Jm. Pela Sito, New Worl, NY 309009 (212) 939—456] 
Sam Roe OSE RB Bra Stap New York, NY 10009 (212) 044-5578 
$bash pb.sh Roe Sam 
Sam Roe Gao RE oth Ste New York, NY LODOS (212) ddd- SGTS 
# When more than one argument is passed to this script, 
#+ it prints *only* the line(s) containing all the arguments. 


Example 35-10. Another (useful) script that recursively calls itself 


Mop pop pppppGÀG:! 
O (o 0» -1 O Oi i5 (). NM P O xo 0 -1 O O1 4 CQ I 


NNN PN 
HSS (es [sS qp 


AS) 
26 
21 


!/bin/bash 
usrmnt.sh, written by Anthony Richardson 
Used with permission. 


usage: usrmnt.sh 


A user with the proper permissions only ha 
usermount /dev/fd0 /mnt/floppy 

instead of 
sudo usermount /dev/fd0 /mnt/floppy 


I use this same technique for all of my 


This is a usermount script that reruns itself using sudo. 


SOUS Lupe 


* sudo scripts, because I find it convenient. 


description: mount device, invoking user must be listed in the 
MNTUSERS group in the /etc/sudoers file. 


If SUDO COMMAND variable is not set we are not being run through 
+ sudo, so rerun ourselves. Pass the user's real and group id 
ab (d[ Zo UD OmeOMMAN DI | 
then 


28 mntusr-$(id -u) grpusr-$(id -g) sudo $0 $* 

29 exit 0 

S9 t3 

SL 

32 4 We will only get here if we are being run by sudo. 
33 /bin/mount $* -o uid-$mntusr,gid-$grpusr 


34 

35 exit 0 

36 

97 Additional notes (from the author of this script): 
38 

38 

40 1) Linux allows the "users" option in the /etc/fstab 
41 file so that any user can mount removable media. 
42 But, on a server, I like to allow only a few 

43 individuals access to removable media. 

44 I find using sudo gives me more control. 

45 

46 2) I also find sudo to be more convenient than 

47 accomplishing this task through groups. 

48 

49 3) This method gives anyone with proper permissions 
50 root access to the mount command, so be careful 
Sd about who you allow access. 

52 You can get finer control over which access can be mounted 
53 by using this same technique in separate mntfloppy, mntcdrom, 
54 and mntsamba scripts. 


® Too many levels of recursion can exhaust the script's stack space, causing a segfault. 
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35.5. "Colorizing" Scripts 


The ANSI [1] escape sequences set screen attributes, such as bold text, and color of foreground and 
background. DOS batch files commonly used ANSI escape codes for color output, and so can Bash scripts. 


Example 35-11. A "colorized" address database 


1 #!/bin/bash 
2 4 ex30a.sh: "Colorized" version of ex30.sh. 
3 4 Crude address database 
4 
E 
6 clear # Clear the screen. 
7 
e€ echo ng ik 
9 echo e "\E(137>44m'"\033 [ImGontact hast \033 Onm 
JL) White on blue background 
11 echo; echo 
12 echo -e "\033[1mChoose one of the following persons: V033[0m" 
L3 Bold 
14 tput sgr0 Reset attributes. 
15 echo "(Enter only the first letter of name.)" 
16 echo 
7 eho sem “Valame sAm" WOS [E iat, N() 3:3: Ohm! Blue 
18 tput sgr0 Reset colors to "normal." 
IS) eelae “wens, iXolbeusel" WU |e ]swgeua, ixoiLetaielY’ 
20 echo em UES Soin! MO \O SS [flim VOSS) [E Orn! Magenta 
All ious sored) 
22 echo "ones, Mildred" 
25 eho -em Y WH 4L ge sem! Y\ OSS) | mS VOSS | Cian! Green 
24 tput sgr0 
25 eho Yniaiicla, wroilaes” 
26 echo -am "NIE e Sain? Y\OSS [Lione OSS: [E Oran!” Red 
al totrt exero) 
26 eho Yans, Morris” 
29 echo 
30 
3l read person 
32 
33 case "$person" in 
34 # Note variable is quoted. 
35) 
36 "LING TN) 
37 # Accept upper or lowercase input. 
36 echo 
39 echo "Roland Evans" 
40 exco VA S21 vies Die 
Zu echo "Hardscrabble, CO 80753" 


echo WY (SOS) 734 98747 

echo "(303) 734-9892 fax" 

echo "revans@zzy.net" 

echo "Business partner & old friend" 


Cr £A am a am SPS a 
© (e) (e Sa) rex Gal dex (i fe» 
- 

- 


echo "Mildred Jones" 

excl. UT1419) in, Tra (Son Noe, d 9 
echo "New York, NY 10009" 
UIS) 555—929. 

echo "(212) 599-9972 ig" 


aoa um 
m= GM 
0) 
Q 
er 
[9] 


55 echo "milliej@loisaida.com" 
56 echo "Girlfriend" 
57 Seine WiBubxeileelesy S Belon Liwy 


58 FE 

59 

60 # Add info for Smith & Zane later. 

61 

62 m 

63 # Default option. 

64 # Empty input (hitting RETURN) fits here, too. 
65 echo 

66 echo "Not yet in database." 

67 p 

68 

69 esac 

70 

Til ieee Seo) # Reset colors to "normal." 
72 

73 echo 

74 

UD exit 0) 


Example 35-12. Drawing a box 


!/bin/bash 
Draw-box.sh: Drawing a box using ASCII characters. 


Script by Stefano Palmeri, with minor editing by document author. 
Minor edits suggested by Jim Angstadt. 
Used in the ABS Guide with permission. 


HH Ht at HH HE EE HE EE HE aE HE aE HE EE EE EE HE aE EE HEE aE EEE aE HE EEE HEE aE aE HE aE HEE EE HEE EEE EH EEH 
## draw_box function doc ### 


The "draw_box" function lets the user 
To (Qeetw mi boz alin al CEMAL s 


Usage: draw_box ROW COLUMN HEIGHT WIDTH [COLOR] 
ROW and COLUMN represent the position 
+ of the upper left angle of the box you're going to draw. 
ROW and COLUMN must be greater than 0 
+ and less than current terminal dimension. 
HEIGHT is the number of rows of the box, and must be > O0. 


Mop p opp ppppÀbÀG: 
O (o 0» -1 O Oi i (). M HP O x 0 -1 O O1 4 CQ I ES 


2.1 HEIGHT + ROW must be <= than current terminal height. 

2 WIDTH is the number of columns of the box and must be > 0. 
225) WIDTH + COLUMN must be <= than current terminal width. 

24 

2,5) E.g.: If your terminal dimension is 20x80, 

26 craw loo» 2 3. 10 4S as. GOV 

AT draw box 2 3 19.45 has bad HEIGHT value (1942 > 20) 

28 draw box 2 3 18 78 has bad WIDTH value (78-43 > 80) 


COLOR is the color of the box frame. 

This is the 5th argument and is optional. 

0-black 1-2red 2=green 3-tan 4=blue 5-purple 6-cyan 7-whit 
If you pass the function bad arguments, 

it will just exit with code 65, 

* and no messages will be printed on stderr. 


Clear the terminal before you start to draw a box. 
The clear command is not contained within the function. 


CO CO CO CO CO CO CO CO CO Dd 
O. =o OF O9 No ESO C XO 
1 
t 


39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Bal 
52 
53) 
54 
55) 
56 
SJ 
58 
5$ 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
vel 
12 
13 
74 
75 
76 
77] 
78 
WS 
80 
81 
82 
83 
84 
9$ 
86 
87 
88 
89 
90 
eut 
22g 
23 
94 
25 
96 
2 
98 
99 
100 
ALO) aL 
102 
103 
104 


# This allows the user to draw multiple boxes, even overlapping ones. 


### end of draw_box function doc ### 
Hatt aa HE aE aE HE EE HE a EE HE EE EE HE aE HE aE EE HE EEE HE aE EE EE HE EE aE HE aE EE aE EE EE HEE EE EE EE EE 


draw_box() { 
HORZ-"-" 


VERT-"|" 
CORNER, CHAR-"«" 


INARGS-4 
E BADARGS-65 
— Mil 
as [| Sue Jie VISIMOUVNRE SY Jg ines # If args are less than 4, exit. 
exit SE BADARGS 
iab 
# Looking for non digit chars in arguments. 


# Probably it could be done better (exercise for the reader?). 


i Cela SE | er -ol [sedes] | te sl [2ebgsEs] || Grab . &S ctew/iowlp than 
exit $E BADARGS 
i3 
BOX_HEIGHT= expr $3 - 1^ # -1 correction needed because angle char "+" 
BOX WIDTH-' expr $4 - 1^ #+ is a part of both box height and width. 
T ROWS-' tput lines' # Define current terminal dimension 
I COLS= tout Cols” #+ in rows and columns. 
aie Si =e 1 ] |i Sl ge ST RONS ip iles # Start checking if arguments 
exit $E BADARGS #+ are correct. 
Tal 
filets $2 =le i 1 |i [ S2 =e Sx COENS g ile 
exit $E BADARGS 
Shae “epgeie Sill sp SBO HELGE ae dL" See ST RONS ||) Chen 
exit $E BADARGS 
£3 
ILE Seon S24 SO D e 1) oe Se OAS |); “then 
exit $E_BADARGS 
IE $3 =ke 2 y p» p 92 xw 4 32 ea 
exit $E BADARGS 
f£ 4 End checking arguments. 
plot char()í # Function within a function. 
echo -e "\E[${1};${2}H"S3 
} 
Seno ne VE SS {So jm! $^ get bos treana Color, 1i (leise. 
# start drawing the box 


count=1 # Draw vertical lines using 
iro (( rsl» Count =9BO MEGAS jet) ye Clo #+ plot char function. 
plor Cmar Sr S2 SVERT 
let count=count+l 
done 
count=1 
c-'expr $2 + $BOX WIDTH' 
ror (((( r=; C@owimNe<=SiOK isin wire erip clo 
plot char Sr Sc SVERT 


105 let count=count+1l 


106 done 

107 

108 count=1 # Draw horizontal lines using 
MS ror (I ep corme eX WED eiia do yo plow (ies romec TOM . 
110 plor «dese Sil $e SISORUS 

1L let count=count+1 

112 done 

113 

114 count=1 

115 z= esq Sil ++ GEN Marat 

E cow ((( G=S25 Corine = SaO NuD (ee: ) A (ele 

117 plot char Sr Sc SHORZ 

LL} let count=count+1 

119 done 

120 

121 plot char Sil S2 SCORN Cig # Draw box angles. 


I 


122 plot char $1 ‘expr $2 + $BOX WIDTH' $CORNER CHAR 
125 plot char exor Sil r GEOR icine S2 SCOIRININIR CHAR 


124 plot char ` expr $1 + $BOX_HEIGHT "expr $2 + $BOX_ WIDTH $CORNER CHAR 
125 

126 echo -ne "\E[Om" # Restore old colors. 
1,277] 

128 P_ROWS= expr $T ROWS — 1' # Put the prompt at bottom of the terminal. 
129) 

130 echo -e "\E[${P_ROWS};1H" 

Sil} 

SZ 

133 

134 # Now, let's try drawing a box. 

135 clear # Clear the terminal. 
136 R=2 # Row 

137 C=3 # Column 

138 H=10 # Height 

139 W=45 # Width 

140 col=1 # Color (red) 

141 chem loe» GR SC Sui SW Seol # Draw the box. 

142 

14/9 este d) 

144 

145 4 Exercise: 

LAG. 4p o——————— 


147 # Add the option of printing text within the drawn box. 


The simplest, and perhaps most useful ANSI escape sequence is bold text, \033[1m ... \033[0m. The \033 
represents an escape, the "[1" turns on the bold attribute, while the "[0" switches it off. The "m" terminates 
each term of the escape sequence. 

bash$ echo -e "\033[1mThis is bold text.\033[0m" 

A similar escape sequence switches on the underline attribute (on an rxvt and an aterm). 


bash$ echo -e "\033[4mThis is underlined text.\033[0m" 


$^; With an echo, the -e option enables the escape sequences. 


Other escape sequences change the text and/or background color. 


bash$ echo -e '\E[34;47mThis prints in blue.'; tput sgr0 


bash$ echo -e '\E[33;44m'"yellow text on blue background"; tput sgr0 


bash$ echo -e '\E[1;33;44m'"BOLD yellow text on blue background"; tput sgr0 


$^, It's usually advisable to set the bold attribute for light-colored foreground text. 


The tput sgr0 restores the terminal settings to normal. Omitting this lets all subsequent output from that 
particular terminal remain blue. 


$^) Since tput sgr fails to restore terminal settings under certain circumstances, echo -ne \E[0m may be a 
better choice. 


Use the following template for writing colored text on a colored background. 


echo -e 'NE[COLOR1;COLOR2mSome text goes here.' 


The "\E[" begins the escape sequence. The semicolon-separated numbers "COLORI" and "COLOR2" 
specify a foreground and a background color, according to the table below. (The order of the numbers does 
not matter, since the foreground and background numbers fall in non-overlapping ranges.) The "m" 
terminates the escape sequence, and the text begins immediately after that. 


Note also that single quotes enclose the remainder of the command sequence following the echo -e. 


The numbers in the following table work for an rxvt terminal. Results may vary for other terminal emulators. 


Table 35-1. Numbers representing colors in Escape Sequences 


Coler — [Foreground Background 


black 30 


Example 35-13. Echoing colored text 


il 
2 
3 
4 
5 
6 
7 
8 


#!/bin/bash 
# color-echo.sh: Echoing text messages in color. 


4 Modify this script for your own purposes. 
# It's easier than hand-coding color. 


black-'NE[30; 47m' 
red-'NE[31;47m' 
green='\E[32;47m' 
yellow='\E[33;47m' 
blue='\E[34;47m' 
magenta='\E[35;47m' 


ps 
C Xo 


[2s 
nd 


ES 
N 


13 cyan='\E[36;47m' 

14 white-'NE[37;47m' 

15 

16 

17 alias Reset="tput sgr0" Reset text attributes to normal 
18 + without clearing screen. 

19 
20 
clc x Color-echo. 

22 Argument $1 = messag 
2:3 Argument $2 = color 
24 { 
25 local default_msg="No message passed." 

26 Doesn't really need to be a local variable. 
27) 
28 message-$(1:-$default msg) Defaults to default message. 

29 color=${2:-Sblack} inet: co lollacik, iit mor SECLE 
30 

Sil ceho coco 

32 echo "Smessage" 

33) Reset # Reset to normal. 

34 

35 return 

36 } 

37 

38 

So 47 iNew, Leris way ale Oules 

40 # 
41 cecho "Feeling blue..." $blue 

42 cecho "Magenta looks more like purple." Smagenta 

43 cecho "Green with envy." Sgreen 

44 cecho "Seeing red?" Sred 

45 cecho "Cyan, more familiarly known as aqua." $cyan 

46 cecho "No color passed (defaults to black)." 

47 # Missing $color argument. 

43) cecho “\Winiiocy\" Color jpassec! (Cleicauilics to blacky, unu 
49 # Empty Scolor argument. 

50 cecho 
Sil # Missing $message and $color arguments. 
By olexelayon WAU puy 

53) # Empty $message and Scolor arguments. 
54 # 
55 
56 echo 

5 

58 exit 0 

59 

60 # Exercises: 

Oi 4 cet 

62 # 1) Add the "bold" attribute to the 'cecho ()' function. 
63 4 2) Add options for colored backgrounds. 


Example 35-14. A "horserace" game 


! /bin/bash 

horserace.sh: Very simple horserace simulation. 
Author: Stefano Palmeri 

Used with permission. 


HERE HEH HEHE EH THERE HH HE EH RR HEE EE EE E E E EE E E H 
Goals of the script: 
playing with escape sequences and terminal colors. 


(69) p py (rl ES Ger Is) j5 


Exercise: 

Edit the script to make it run less randomly, 

+ set up a fake betting shop 

Ui 5 oe 6 Pun 3 $5 g WES Sweetie (EO wemime ms Que a mowie 


script gives each horse a random handicap. 
The odds are calculated upon horse handicap 

+ and are expressed in European(?) style. 

E.g., odds=3.75 means that if you bet $1 and win, 
+ you receive $3.75. 


IN TSS Se ee ee et 
O Woy (oe) SJ) vony Gal dex «o9 IS) Sy Wo) 
i 
(or 

[0] 


2 The script has been tested with a GNU/Linux OS, 

22 #+ using xterm and rxvt, and konsole. 

BS) On a machine with an AMD 900 MHz processor, 

24 #+ the average race time is 75 seconds. 

25) On faster computers the race time would be lower. 

26 So, if you want more suspense, reset the USLEEP ARG variable. 
2 

28 Script by Stefano Palmeri. 

29 HHEREEE EEE HE HEH HE HE EEE EE HH HE HE ERE EE HE EEE HE ERE EE EE EE EE ERE HE HEE 
30 

31 E_RUNERR=65 

32 

33 Check if md5sum and bc are installed. 

34 if ! which bc &> /dev/null; then 

35 echo bc is not installed. 

36 echo Wein aem e uu. 

37 exit $E RUNERR 

JG EL 

39 if ! which md5sum &» /dev/null; then 

40 echo md5sum is not installed. 

41 exeleg) WiOEu NE sevice 5 y V 

42 exit $E RUNERR 

43, Ea 

44 

45 Set the following variable to slow down script execution. 

46 It will be passed as the argument for usleep (man usleep) 

47 #+ and is expressed in microseconds (500000 = half a second). 

48 USLEEP ARG-0 

49 

50 Clean up the temp directory, restore terminal cursor and 

Gil gar eenia Colors —= ik eernog Hacerte oy Cel- 

52 trajo echo -em Ar 251: echo -m "WEEDS" E. Srey GENGIN 

53. OME up 20 Op am =i SROS BUNC MMS DIR. TERM EXIT 

54 S the chapter on debugging for an explanation of 'trap.' 

55 

56 Set a unique (paranoid) name for the temp directory the script needs. 
57 HORSE RACE TMP DIR-S$HOME/.horserace-' date +%s`-`head -c10 /dev/urandom V 
58 md5sum | head -c30' 

59 

60 Create the temp directory and move right in. 

61 mkdir S$HORSE RACE TMP DIR 

62 cd SHORSE RACE TMP DIR 

63 

64 

65 # This function moves the cursor to line $1 column $2 and then prints $3. 
66 4$ E.g.: "move and echo 5 10 linux" is equivalent to 

67 #+ "tput cup 4 9; echo linux", but with one command instead of two. 
68 # ote: "tput cup" defines 0 0 the upper left angle of the terminal, 
69 #+ echo defines 1 1 the upper left angle of the terminal. 

70 move and echo() { 

71 echo -ne "\E[S{1};S{2}H"™"S3" 

M 

pe 


74 # Function to generate a pseudo-random number between 1 and 9. 


fs 
«o 


0 


random. 1-9 


( 
head -c10 /dev/urandom 

} 

# 

draw_horse_one() { 

Seine, ay WU 
} 
draw_horse_two() { 


echo -n " 


4 Defin 
IN GouS-' tu cols 
N LINES- tput lines” 


# Need at least a 20-LIN 


sur [| SECOS Xt GO | 
echo "'basename $0` 
echo 
exit $ 
1E 3L 


ERR 


E RUN 


| md5sum | 


"//$MOVE. HORS 


rac 


[ez | 


E// 


"\\\\SMOVE_HORS 


TANAY 


current terminal dimension. 


ES X 80-COLUMNS 
[ SNES Ie 


# Start drawing the race field. 


# Need a string of 80 chars. 


BLANK80=" seq -s LOO | 


clear 


See below. 


head -c80` 


terminal. 
20 vee than 


# Set foreground and background colors to white. 


IAES V pAn 


echo -ne 


# Move the cursor on the upper left angle of the terminal. 


Toure eup ©) ©) 


# Draw six white lines. 
itoie im am “seep S B. clo 

echo SBLANK80 
done 


# Sets foreground color to black. 


echo -ne 'NE[30m' 
move and echo 3 1 "START 1" 
move and echo 3 75 FINISH 
move and echo 1 5 "|" 
move and echo 1 80 "|" 
move and echo 2 5 "|" 
move and echo 2 80 "|" 
move and echo 4 5 "| UU 
move and echo 4 80 "|" 
move and echo 5 5 "V 3j" 
move and echo 5 80 "v" 


# Set foreground color to red. 


Scho ne Va eim 

# Some ASCII art. 

move and echo 1 8 "..@@@..@@@@@...@@@@e. 
movesandtecho 2. Ter MEO or s (B scene norme Q. 
Mowe cual exeleg 3 i Wee s css ss siois (dr 
mMoverand exclave, A. Yas 6 stb e cs con Q. 


Gonos ses cn 
s@aedl@sGoans Gag 
.QCCCC.CCCR.... 
SOO c ooo wo 


Two functions that simulate "movement," when drawing the horses. 


Check wate 


meedseancU>Colse x 20- lames mt esaet 


"Your terminal is ${N_COLS}-cols X ${N_LINES}—-lines." 


# Use the 80 chars string to colorize the terminal. 


141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
dL 
152 
15:3) 
154 
155 
156 
115 
158 
1159 
160 
161 
162 
163 
164 
165 
166 
1.6) 7 
168 
169 
170 
yal 
172 
LHS) 
174 
LHS) 
176 
17/3) 
178 
LAS 
180 
181 
182 
183 
184 
185 
186 
187 
188 
189 
190 
gil 
192 
193 
194 
195 
196 
197 
198 
198; 
200 
201 
202 
203 
204 
205 
206 


ile we uel exeo 5 m Usos cs socsonc Caeo Ene o ea o CGO, 5.0 
move and echo 1 43 "QCCC... CCA... CACC.. CAAC.. CCC." 
movezand echon TAS aM a Chetan [dE " 
move and echo 3 43 "@@@@..@@@@@.@..... @@@@...@@@.." 
Inowemencmecloma “sy aee (dude (oM (dete tan: @.¥ 

5) 


43 "Q...0.0...G0..00800..G0000.G0000.." 


move and echo 


# Set foreground and background colors to green. 
echo -ne '\E[32;42m' 


# Draw leven green lines. 
Sate (Up 5 0) 
igi im am eea iil g co 
echo SBLANK80 
done 


# Set foreground color to black. 
echo -ne 'NE[30m' 
Caine Cus S» (0 


# Draw the fences. 
echo MHHFAFFFAFFFAFFFFFFFEFEFFEFEFFEFEFFEF4E4F4F444\ 
THER TTYATYTTYLEFTTTYTTETLEAAABBeAAXSMHU 


iSgiuuE, Cujo Is) (0) 
echo. "4 EE ETE T T E E EH ESESESE pee eee ETEEN 
mcccccccccc: c f f c c: f f c d c afl 


# Set foreground and background colors to white. 
echo -ne 'NE[37;47m' 


# Draw three white lines. 

oie im am “seo SP co 
echo SBLANK80 

done 


# Set foreground color to black. 
echo -ne '\E[30m' 


# Create 9 files to stores handicaps. 
ior im am “see 10 7 68° p? clo 

touch $n 
done 


# Set the first type of "horse" the script will draw. 
HORSE_TYPE=2 


# Create position-file and odds-file for every "horse". 
#+ In these files, store the current position of the horse, 
#+ the type and the odds. 
itgic JUN aim "meer YR clo 
touch horse SHENI position 
touch odds S(HN) 
echo \-1 > horse $(HN]. position 
echo S$HORSE TYPE >> horse S(HN) position 
# Define a random handicap for horse. 
HANDICAP-'random 1 9" 


# Check if the random 1 9 function returned a good value. 


while ! echo SHANDICAP | grep [1-9] &» /dev/null; do 
HANDICAP-'random 1 9" 
done 
# Define last handicap position for horse. 
LHP-'expr SHANDICAP \* 7 + 3^ 
coe imam abe ^e LO 7 Gus" s clo 
echo SHN >> SFILE 


[SOF INS) N IST SY INS) ISS) INS) 


N 
«o 


done 
# Calculate odds. 
case SHANDICAP in 
1s) ODDS—Vechow SHANDICAP I NASON 255 tele 25. be 
echo SODDS > odds_${HN} 
2 | 399. ODDS ceo SubWDI(CANS Wes (520 43 1,25 | 19€ 
echo SODDS > odds S(HN) 
4 | 5 | ©) ODDS- eco SHIANIDICAR W^ O55 4 1.25 | le 
echo SODDS > odds S(HN) 
7 | 83 GIDES- echo SANDIN ya (0.95 4 1.25 | lee 
echo SODDS > odds S RNI 
9)) ODDS= exem. SKANDICAS X (0,90 «s 1,25 | Be 
echo SODDS > odds S(HN) 
esac 
done 
# Print odds. 


joxsibavE evgleler() if 
wote «eb (5 O 
echo -ne 'NE[30; 42m' 
for Bil 3um "mee Sp coe 
echo "#SHN odds->" ^cat odds S$(HNJ)' 


done 


) 


# Draw the horses at starting line. 
draw horses() { 
wigte CLS 6 9) 
echo na '\E[30;42m' 
tor iN am “see 9 s clo 

echo /NNSHN/NN" " 


done 

} 

joe Lime, (exelels: 

echo -ne 'NE[47m' 


# Wait for a enter key press to start the race. 


# Th Scape sequence 'NE[?251' disables the cursor. 
iejowne. eca 17 0) 

exciso) =e Wa P25il press [exea] kay tO start Cne Yee oc 
read -s 

# Disable normal echoing in the terminal. 


# This avoids key presses that might "contaminate" the screen 
#+ during the race. 
stty -echo 


# 
# Start the race. 


draw_horses 

echo -ne '\E[37;47m' 
move and echo 18 1 SBLANK80 
echo -ne 'NE[30m' 

move and, echo 18 1 Starting... 
sleep 1 


2/738). ur SSE CAS Conima Cie jel sess Sla Jugis 
274 WINNING POS-74 

2175 

276 # Define the time the race started. 
277 START TIME-' date +%s~ 


278 

279 # COL variable needed by following "while" construct. 

280 COL-0 

281 

282 while [ $COL -lt $WINNING POS ]; do 

283 

284 MOVE HORSE-0 

285 

286 # Check if the random 1 9 function has returned a good value. 
287 while ! echo $MOVE HORSE | grep [1-9] &> /dev/null; do 
288 MOVE_HORSE=` random_1_9` 

289 done 

290 

291 # Define old type and position of the "randomized horse". 
292 HORS EMNIVERN TECI ERRORS EC _S MONAT, JO OEE POS HERON || iEmaLl al dL 
DS) COL=$ (expr "cat horse_${MOVE_HORSE}_position | head -m 1°) 
294 

295 ADD_POS=1 

296 # Check if the current position is an handicap position. 
297 if seq 10 7 68 | grep -w $COL &> /dev/null; then 

298 if grep -w $MOVE HORSE $COL &> /dev/null; then 

2 99) ADD POS-0 

300 grep -v -w  $MOVE HORSE $COL > $(COL) new 
301 xam =I SCOR 

302 mv -f ${COL}_new $COL 

303 else ADD POS-1 

304 if a 

305 else ADD_POS=1 

306 teat 

307 COL=*expr $COL + $ADD POS' 

308 echo $COL > horse S$(MOVE HORSE) position # Store new position. 
309 

S10) # Choose the type of horse to draw. 

JLI case SHORSE TYPE in 

312 1) HORSE_TYPE=2; DRAW_HORSE=draw_horse_two 

SLS Pi 

314 2) HORSE TYPE-1; DRAW HORSE-draw horse one 

SILS esac 

31.65 echo SHORSE TYPE >> horse S(MOVE HORSE). position 

3t y # Store current typ 

318 

SL) # Set foreground color to black and background to green. 
320 echo -ne '\E[30;42m' 

SA AL 

227 # Move the cursor to new horse position. 

323 tput cup 'expr $MOVE HORSE + 5^ \ 

324 "cat horse $(MOVE HORSE) position | head -n 1° 

SIZI5 

326 # Draw the horse. 

7» SDRAW HORSE 

328 usleep SUSLEEP_ARG 

329 

330 # When all horses have gone beyond field line 15, reprint odds. 
SO touch fieldlinel5 

332 a | SCOl = 45 Je Then 

333) echo $MOVE HORSE >> fieldlinel15 

334 ÍE3L 

5] 95) if [ we -X fieldilinel5 | cut -fl -d " " = 9 |]; then 
336 print, odds 

SS 2 X ENE eiae 


338 i ak 


33g 
340 
341 
342 
343 
344 
345 
346 
347 
348 
349 
350 
351 
352 
393 
354 
355 
356 
35 
358 
39$ 
360 
361 
362 
363 
364 
3095 
366 
367 
368 
369 
370 
371 
372 
373 
374 
375 
376 
377 
378 
379 
380 
SEL 
382 
383 
384 
38.5 
386 
SET 
388 
389 
390 


# Define the leading horse. 


EILGRIAS I IOS" CHE 'qpesodExem || Sort =m | tani =i" 


# Set background color to white. 


echo -ne '\E[47m' 
tunc, cup Ly Y 


cho n Current leader: grep 
" " 


done 


# Define the time the race finished. 
FINISH TIME-' date +%s~ 


# Set background color to green and enabl 


echo -ne '\E[30;42m' 
echo -en '\E[5m!' 


# Make the winning horse blink. 
iow Clie "edo SIMOWIT_ISOIRGIN wp 5) \ 


cat horse S$(MOVE HORSE) position | head 


SDRAW HORSE 


# Disable blinking text. 
echo -en '\E[25m' 


blinking text. 


=m ^ 


# Set foreground and background color to white. 


echo oe '\E[37;47m' 
move and echo 18 1 SBLANK80 


# Set foreground color to black. 
echo -ne 'NE[30m' 


# Make winner blink. 
gote (hup 17 © 
echo -e "\E[5mWINNER: $MOVE HORSENE[25m"" 


Odds: ^cat odds, S(MOVE HORSE 


1 Race times eaor SIPING TIME — SCAR TIME secs” 


# Restore cursor and old colors. 
echo. em IE 2b" 
echo -en "NE[Om" 


# Restore echoing. 
stty echo 


# Remove race temp directory. 
rm -rf SHORSE_ RACE TMP DIR 


tput cup 19 0 


exit 0 


w SHIGHEST POS *position | cut -cY7^wN 


See also Example A-21, Example A-44, Example A-52, and Example A-40. 


® There is, however, a major problem with all this. ANSI escape sequences are emphatically non-portable. 
What works fine on some terminal emulators (or the console) may work differently, or not at all, on 
others. A "colorized" script that looks stunning on the script author's machine may produce unreadable 
output on someone else's. This somewhat compromises the usefulness of colorizing scripts, and possibly 
relegates this technique to the status of a gimmick. Colorized scripts are probably inappropriate in a 
commercial setting, i.e., your supervisor might disapprove. 


Moshe Jacobson's color utility (http://runslinux.net/projects.html#color) considerably simplifies using ANSI 
escape sequences. It substitutes a clean and logical syntax for the clumsy constructs just discussed. 


Henry/teikedvl has likewise created a utility (http://scriptechocolor.sourceforge.net/) to simplify creation of 
colorized scripts. 


Notes 


[1] ANSIis, of course, the acronym for the American National Standards Institute. This august body 
establishes and maintains various technical and industrial standards. 
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35.6. Optimizations 


Most shell scripts are quick 'n dirty solutions to non-complex problems. As such, optimizing them for speed is 
not much of an issue. Consider the case, though, where a script carries out an important task, does it well, but 
runs too slowly. Rewriting it in a compiled language may not be a palatable option. The simplest fix would be 
to rewrite the parts of the script that slow it down. Is it possible to apply principles of code optimization even 
to a lowly shell script? 


Check the loops in the script. Time consumed by repetitive operations adds up quickly. If at all possible, 
remove time-consuming operations from within loops. 


Use builtin commands in preference to system commands. Builtins execute faster and usually do not launch a 
subshell when invoked. 
Avoid unnecessary commands, particularly in a pipe. 


cat "$file" | grep "Sword" 


il 
2 
grep USivouecl! Ui eU 
4 
3 


# The above command-lines have an identical effect, 
6 #+ but the second runs faster since it launches one fewer subprocess. 


The cat command seems especially prone to overuse in scripts. 


Use the time and times tools to profile computation-intensive commands. Consider rewriting time-critical 
code sections in C, or even in assembler. 


Try to minimize file I/O. Bash is not particularly efficient at handling files, so consider using more 
appropriate tools for this within the script, such as awk or Perl. 


Write your scripts in a modular and coherent form, [1] so they can be reorganized and tightened up as 
necessary. Some of the optimization techniques applicable to high-level languages may work for scripts, but 


others, such as loop unrolling, are mostly irrelevant. Above all, use common sense. 


For an excellent demonstration of how optimization can dramatically reduce the execution time of a script, see 
Example 16-47. 


Notes 


[1] This usually means liberal use of functions. 
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35.7. Assorted Tips 


35.7.1. Ideas for more powerful scripts 


e. 
You have a problem that you want to solve by writing a Bash script. Unfortunately, you don't know 
quite where to start. One method is to plunge right in and code those parts of the script that come 
easily, and write the hard parts as pseudo-code. 


!/bin/bash 


ARGCOUNT=1 # Need name as argument. 
E_WRONGARGS=65 


if [ number-of-arguments is-not-equal-to "SARGCOUNT" ] 


AKRAKAKRARAKRARAAKAKRAAKAKAKRKRKR KRAKAKRKAKAKRKRAARARAARANR 


Can't figure out how to code this 
3 5 o a SO WINES ie alin peeucloq—cocks. 


echo "Usage: name-of-script name" 
# BEEN E a ian KE More pseudo-code. 
exit SE_WRONGARGS 


NPRPRPPRP PPP PY 
O (o 00 -1 O Ui i (). M e (o 0 -1 O O1 i Q Io E 
. ct 
op 
0) 
5 


exit 0 
A 
22 # Later on, substitute working code for the pseudo-code. 
23 
24 # Line 6 becomes: 
25 if [ $4 -ne "SARGCOUNT" ] 
26 
27 4 Line 12 becomes: 


28 echo "Usage: 'basename $0` name" 
For an example of using pseudo-code, see the Square Root exercise. 
e. 
To keep a record of which user scripts have run during a particular session or over a number of 
sessions, add the following lines to each script you want to keep track of. This will keep a continuing 
file record of the script names and invocation times. 


# Append (>>) following to end of each script tracked. 


i 

2 

3 whoami>> S$SAVE FILE 
4 echo SÍ SIS TILE 
5 date>> S$SAVE FILE 
6 
7 
8 
9 


User invoking the script. 
Script name. 

Date and time. 

Blank line as separator. 


de Sb db dk 


echo>> $SAVE FILE 


# Of course, SAVE FILE defined and exported as environmental variable in -/.bashrc 
#+ (something like -/.scripts-run) 


The >> operator appends lines to a file. What if you wish to prepend a line to an existing file, that is, 
to paste it in at the beginning? 


1 file-data.txt 

2. igaedber was: abe. Tree. eiela line (ur cera ieee iaLllgyeeom Y 
3 

4 echo $title | cat Suede >$rile mew 


5 i; Yee —" concatenetes elem to Grile, 

6 # End result is 

7 #+ to write a new file with $title appended at *beginning*. 
This is a simplified variant of the Example 19-13 script given earlier. And, of course, sed can also do 
this. 


e. 
A shell script may act as an embedded command inside another shell script, a Tcl or wish script, or 
even a Makefile. It can be invoked as an external shell command in a C program using the 
system () call, ie. system("script name");. 

e. 


Setting a variable to the contents of an embedded sed or awk script increases the readability of the 
surrounding shell wrapper. See Example A-1 and Example 15-20. 


Put together files containing your favorite and most useful definitions and functions. As necessary, 
"include" one or more of these "library files" in scripts with either the dot (.) or source command. 


il SCRIPT LIBRARY 

2 

3 

4 Note: 

E No "£!" here. 

6 No "live code" either. 

7 

8 

9 Useful variable definitions 

10 

11 ROOT_UID=0 # Root has SUID O0. 

12 E NOTROOT-101 # Not root user error. 

13 MAXRETVAL-255 # Maximum (positive) return value of a function. 
14 SUCCESS=0 

15 FAILURE--1 

16 

1L7/ 

18 

19 # Functions 

20 

21 Usage () # "Usage:" message. 

22 

23 mie [p x Wis] # No arg passed. 

24 then 

25 msg=filename 

26 else 

20] msg=S@ 

28 dE al 

219 

30 echo "Usage: basename SO "$msg"" 

Si} 

32 

33 

S34 Cheek ait root (0) # Check if root running script. 
33 4 # From "ex39.sh" example. 
36 if | Vourn me VEROON UID™ | 

37 then 

38 echo VIMPISIE IOS roor Co icum. Cais GOLPE,” 

39 exit SE NOTROOT 

40 $E 3l 

41 ) 

42 

43 

44 CreateTempfileName () # Creates a "unique" temp filenam 
a5 fi # From "ex51.sh" example. 


46 prefix-temp 
47 suffix= eval date +%s° 


48 
49 
50 
Sil 
52 
53 
54 
E 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
33 
72 
13 
74 
35 
76 
p 
78 
HS 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
ESI 
92 
ES 
94 
95 
96 
2 
98 
99 
100 
101 
102 
103 
104 
LOS 


Tempfilename=Sprefix.$suffix 


isalpha2 () 
{ 

[Ste Seer al dui 
case $1 in 
*[l'a-zA-Z]*|"") 
*) return $SUCCI 
esac 


O 


abs 


{ 


EFARGERR= 99999 


ai | 
then 


USH ] 


=a 


EG 


re 
ESS 


9 


return $E ARG 


ERR 


feu 


siae (I 
then 
absval-$1 
else 
let 
iE, 


"SI" -ge 0 


"absval 


return Sabsval 


tolower () 


{ 


am Al 

then 
echo "(null)" 
return 

ipa 


WSW ] 


anes 


wea | 


echo 


return 


For example: 


echo 


] 


Exercis 


) 


Rewrit 
Lo Uppercase 


# Tests whether *entir 
# From "isalpha.sh" example. 
turn SFAILURE 


SFAILURE; 


UR 


EGIA 
Pi 


trank One: 


Absolute value. 
Caution: 


Need arg passed. 


If non-negative, 


Stays. BSS, 
Otherwise, 
change sign. 


# Converts string(s) 
to lowercase. 


# If no argument (s) 
send error messag 


passed, 


Max return value 


string* is alphabetic. 


ASS) 


Obvious error value returned. 


passed as argument (s) 


(C-style void-pointer 
and return from function. 


wie A A ees 
# Translate all passed arguments 


($8). 


oldvar="A seT of miXed-caSe LEtTers" 
newvar= tolower 
"Snewvar" 


So llickwaie 7 
# a set of mixed-case letters 


rror message) 


Use command substitution to set a variable to function output. 


this function to change lowercas 
toupper () [easy]. 


passed argument (s) 


Use special-purpose comment headers to increase clarity and legibility in scripts. 


1 ## Caution. 
2. Sei IEE TATA 


Ht 


4 


The "-rf" options to "rm" are very dangerous, 
3 ##+ especially with wild cards. 


14 


O 


Line continuation. 


Weak} Lej 


of a multi-line comment, 


lise. di 


amel EMiS sis, cle inset disbiouers 


Norte. 


Pist item. 


LS wikile |p Wew 


arl" != 


> Another point of view. 


"end" ] 


#> while test 


usvan bhu ene 


Dotan Barak contributes template code for a progress bar in a script. 


Example 35-15. A Progress Bar 


NPRPRPPRP PPP PY 
SCOMIKDTHRWNHFPOWOMAIDGURWNHE 


WWNNNNNNN DN NH 
[= (€» (we) (o9 c ix, (a de (» fw» (3 


w 
N 


C90 CO CO CO CO 
cu (x. € dme» 


38 


i tU 


bin/bash 


# progress-bar.sh 


# A 


BAR 
BAR 
BAR 
BAR 
BAR 
BRA 
LI 


uthor: Dotan Barak 
# Used in ABS Guide with permission 


_WIDTH=5 
ac 
me 
me 
CHAR_FU 


0 


HAR START-"[" 
HAR END-"]" 
HAR EMPTY-"." 


LL2"-" 


CKET CHARS-2 


IT-100 


print progress bar() 


{ 


M 


let 


let 


For 


don 


for 


don 


(very minor revisions by ABS Guide author). 


Mew doabub = 


"empty_limit 


((370; 
e 


((370; 


bar line-"$[(bar line)$(BAR CHAR | 


e 


Prepare the bar. 
bar line-"$(BAR CHAR START)" 


((($1 


Calculate how many characters 


(thanks!). 


($1 — SBRACKET CHARS) 


jesse EE e 
bar lines ts {oar line)])$(BAR CHAR FULL}" 


j«empty limit; 


jew) Clo 


jira) ye ciue 


bar_line="$ {bar_line}${BAR_CHAR_END}" 


D 
$6 


$s" 


$2 Siar lae 


39 # Here is a sample of code that uses it. 


40 
41 
42 
43 
44 
45 
46 
47] 


MAX 
EOE 


_PERCENT 
( (i20; 


=100 
i<=MAX_P 


ERC 


ENT; 


usleep 10000 


it++)); 


do 


Or run some other commands 


print progress bar ${BAR_WIDTH} $(i) 


ech 


o -en "N 


r" 


Calculate how many characters will be full. 
— SBRACKET. CHARS) 


v S2) J Sup) A 


will be empty. 


= UL obti j; V 


EMPTY)" 


48 done 

49 

50) euis WY 
51 

52 exit 


A particularly clever use of if-test constructs is for comment blocks. 


#!/bin/bash 


COMMENT_BLOCK= 
# Try setting the above variable to some value 
#+ for an unpleasant surprise. 


if [ $COMMENT BLOCK ]; then 


Comment block -- 


[Es 


This is a comment line. 
This is another comment lin 
This is yet another comment line. 


= 


e 


(= [= s 
=<] ep) (il des (03. I [3 9 We) (ex cL ey) (Oap gEx (93. [853 [5 


Seko Amais wL ine SEN, ^ 


Comment blocks ar rror-free! Whee! 


Eal 


echo "No more comments, please." 


[3 [SS [RS [RS d gp 
[Sr [sy [ Ct» (er (O9 


24 exit 0 
Compare this with using here documents to comment out code blocks. 


Using the $? exit status variable, a script may test if a parameter contains only digits, so it can be 
treated as an integer. 


1 #!/bin/bash 

2 

3 SUCCESS=0 

4 E BADINPUT-85 

5 

6 cest Uglu ne Q -o VSI" ec 0 22/ck md 

p An integer is either equal to 0 or not equal to O0. 
8 2>/dev/null suppresses error messag 

9 

IO ai | $99 =ne VSSUCCESS” ] 

11 then 

12 echo "Usage: '"basename $0° integer-input" 

1,3 exit $E BADINPUT 

Wa a 

LS 

16 let "sum = $1 + 25" # Would give error if $1 not integer. 
17 echo "Sum = $sum" 

18 

19 # Any variable, not just a command-line parameter, can be tested this way. 
20 
21 exit 0 


e The 0 - 255 range for function return values is a severe limitation. Global variables and other 
workarounds are often problematic. An alternative method for a function to communicate a value 
back to the main body of the script is to have the function write to st dout (usually with echo) the 


"return value," and assign this to a variable. This is actually a variant of command substitution. 


Example 35-16. Return value trickery 


#!/bin/bash 
# multiplication.sh 


multiply () # Multiplies params passed. 
{ # Will accept a variable number of args. 


local product-1 


Winall | eg VS T # Until uses up arguments passed... 
do 
lec Yprocuct “= Si." 
SONTE 
done 
echo $product # Will not echo to stdout, 
} #+ Since this will be assigned to a variable. 
mlel 1859 55^ mult 2-215 2 il 
vall= multiply $multl1 $mult2^ 


NPRPRPPRP PPP PY 
O io 00 -1 OY O1 i$ (0. M IS. O xo 0 -1 O O1 4S CQ) IO S 


echo "$multi X Smult2 = $vall" 
# 387820813 


NNN 
WDE 


multi-25; mult2=5; mult3-20 


24 val2-' multiply $multi $mult2 $mult3^ 

25 eno emele 3€ Suede x walls = S3uwedi2" 

26 # 2500 

2) 

28 mult1-2-188; mult2=37; mult3-25; mult4=47 

29 val3-' multiply $multl1 $mult2 $mult3 $mult4' 

30 dcxelowo “Sumi xX gmot OX Simpulies X fSunudbed = Sweulse 
al # 8173300 

32 

33 erie V 


The same technique also works for alphanumeric strings. This means that a function can "return" a 
non-numeric value. 


) 


1 capitalize ichar () Capitalizes initial character 
2 { + of argument string(s) passed. 
3 

4 string0="Se" Accepts multiple arguments. 

E 

6 firstchar=${string0:0:1} Hirst eharacter. 

7 stringl-$(string0:1) REGIE Oi sm (S) c 

8 

9 FirstChar= echo "Sfirstchar" | tr a-z A-Z' 

LO Capitalize first character. 

iI 

12 echo WW SinslieGhe Clone osti meilt QUEUE to secota 

13 

L4 

[5 


16 newstring= capitalize_ichar "every sentence should start with a capital letter." 
17 echo "$newstring" # Every sentence should start with a capital letter. 


It is even possible for a function to "return" multiple values with this method. 


Example 35-17. Even more return value trickery 


e 


PPP PR + Ge 
O00 -1 OY Ui 4 (). 9. EB. O (0 00 -1 O Or iS (QN S 


Nh +t 
CQ Xo 


NO NO BO PO NO PO P2 
sup cx; (Ga) dE Gs) soy qp 


N 
[99] 


#!/bin/bash 
# sum-product.sh 
# A function may "return" more than one value. 


sum_and_product () # Calculates both sum and product of passed args. 
{ 

echo S(( Si + 9S2 )) S(( Sl = $2 J) 
# Echoes to stdout each calculated value, separated by space. 


echo 
echo "Enter first number " 
read first 


echo 
echo "Enter second number " 
read second 


echo 

retval= súm and product $first $second' # Assigns output of function. 
sum-'echo "$retval" | awk '{print $1)" # Assigns first field. 
product-'echo "$retval" | awk '{print $2)" # Assigns second field. 


echo "Sfirst + $second = $sum" 
echo "Sfirst * Ssecond = Sproduct" 
echo 


exec O 


There can be only one echo statement in the function for this to work. If you alter the previous 


example: 
1 sum_and_product () 
2 { 
3 echo "This is the sum_and_product function." # This messes things up! 
4 Scho S(( Sil + S2 )) S(( SL * S2 n) 
I) 
S 
7 retval-'sum and product $first $second' # Assigns output of function. 
8 # Now, this will not work correctly. 


Next in our bag of tricks are techniques for passing an array to a function, then "returning" an array 
back to the main body of the script. 


Passing an array involves loading the space-separated elements of the array into a variable with 
command substitution. Getting an array back as the "return value" from a function uses the previously 
mentioned strategem of echoing the array in the function, then invoking command substitution and 
the ( ... ) operator to assign it to an array. 


Example 35-18. Passing and returning arrays 


=| cx; (Ga) des (bo es) JI 


#!/bin/bash 
# array-function.sh: Passing an array to a function and 
# "returning" an array from a function 


Pass_Array () 


{ 


(so: «| (x (Gn de. GO) WS) ft] &] wer (9 


I-A 


(C3) TES [RS (o [mor ISS) dmSy ISS) eer IS) TES OY D 
i= (&». We) (os. x ey Ga de (5 [IS ( €» we 


C0 CO CO CO CO CO CO CO 
We) (we c] (x, (On de TGS) (S 


D 
e 


(Ou Gal gS GS. EN S ese SS TS, GS dits 
[— @) oy (o9) c fexy (On| ES ED TS) 1 


(Onl (On (np (Gm) Gah Gab (Om. (Om 
We) (Ger (x Gal HES (eo) [R5 


(Od) fey) KON Ten), Tony “ny 
@y Cl des (65 Is [V oS) 


NYY DA Oo 
i= (C3 Wey (9 I] 


~~ 
Wh 


local passed_array # Local variable! 


passed_array=( “echo "$1" 


echo "S{passed_array[@]}" 
# List all the elements of the new array 
#+ declared and set within the function. 


original array-( elementl element2 element3 element4 element5 ) 
echo 
echo "original array = S{original_array[@]}" 


Pack a variable 


List all elements of original array. 


aUas tS deles TEIN eiar joSINNIES Dessine ein chare Co Gl iEUUACIE MOM . 
ck ck Ck ck ck ck ck Ck ck ck Ck ck kk ck kk kk ck kk ko ko Sk Mk ko KKK KKK 


argument= echo $[(original array[8]])' 
ck ck ck ck ck ck ck ck ck ck ck ck kk ck ck ok ck EEr RE kkk kk kR k Ek 


+ with all the space-separated elements of the original array. 


Attempting to just pass the array itself will not work. 


This is the trick that allows grabbing an array as a "return value". 
Ck ck ck ck ck ck ck Ck ck ck Ck ck ck ck ck Ck ck ck ck ck kk ck ck ok kk ck ko Sk kk ck Mk Sk kx kx ko ko. 


returned array-( ^Pass Array "$argument"' ) 
Lax dx d d dd KKK KKK KKK KKK dd d dd d d dd dd d d dd d d d dd 


Assign 'echoed' output of function to array variable. 


echo "returned array = S${returned_array[@]}" 


choki 
Now, try it again, 


Pass Array "Sargument" 


echo 


* attempting to access (list) 


the array from outside the function. 


The function itself lists the array, but 

+ accessing the array from outside the function is forbidden. 
echo "Passed array (within function) = S{passed_array[@]}" 

NULL VALUE since the array is a variable local to the function. 


aE HHH HHT EH AE FE AE FE FE HE HE HE EEE 


# And here is an even more explicit example: 


ret_array () 

{ 
for element in {11..20} 
do 


echo "Selement " # Echo individual elements 
done #+ of what will be assembled into an array. 
} 
arr=( $(ret array) ) # Assemble into array. 


Selo: WCajocwueiing, euer N Varre erom EONO ieee acrady (0) ooo 
Scho Une slanem: QUE array \Weuce\ as Stair [E21] sw # 13 (zero-indexed) 


echo -n "Entire array is: 
echo S{arr[@] } 


echo 


@ dd 12 i15 14 35 16 27 1$ 19 20 


74 
75 es 0) 


For a more elaborate example of passing arrays to functions, see Example A-10. 

e 
Using the double-parentheses construct, it is possible to use C-style syntax for setting and 
incrementing/decrementing variables and in for and while loops. See Example 11-12 and Example 
11:17. 

e. 
Setting the path and umask at the beginning of a script makes it more "portable" -- more likely to run 
on a "foreign" machine whose user may have bollixed up the $PATH and umask. 


#!/bin/bash 
PATH=/bin:/usr/bin:/usr/local/bin ; export PATH 
umask 022 # Files that the script creates will have 755 permission. 


(ni dEx (es) ISS) [ 


i? Tacme co tam Da Allem, Ler Elis tids 
e 


A useful scripting technique is to repeatedly feed the output of a filter (by piping) back to the same 
filter, but with a different set of arguments and/or options. Especially suitable for this are tr and grep. 


1 # From "wstrings.sh" example. 


2 
3 qligt= strings VS | jee Mom a-z || te "f eseeessl" £z | \ 
4 iue =e Balona] X || ice e ALAST dw || wie ms v v^ 


Example 35-19. Fun with anagrams 


1 #!/bin/bash 

2 4 agram.sh: Playing games with anagrams. 

3 

4 # Find anagrams of... 

5 LETTERSET-etaoinshrdlu 

EEINISIBISEIE RE : How many letters minimum? 

7 # 1234567 

8 

9 anagram "SLETTERSET" | Find all anagrams of the letterset... 
10 grep "SEILTER" | With at least 7 letters, 

J3L pee "^m" | SESE ios) ubl Uns 

12 greo -v ee" | no plurals 

13 grep -v 'ed$' no past tense verbs 

14 # Possible to add many combinations of conditions and filters. 
15 

16 # Uses "anagram" utility 

17 #+ that is part of the author's "yawl" word list package. 
18 # http://ibiblio.org/pub/Linux/libs/yawl-0.3.2.tar.gz 
19 4 http://bash.neuralshortcircuit.com/yawl-0.3.2.tar.gz 
20 
All exane (0) # End of code. 
22 
23 
24 bash$ sh agram.sh 
25 islander 
26 isolate 
27 isolead 
28 isotheral 
29 
30 
Sil 


CO 
N 
+ 


Exercises: 


ei s mme 

34 # Modify this script to take the LETTERSET as a command-line parameter. 
35 4 Parameterize the filters in lines 11 - 13 (as with S$FILTER), 

36 #+ so that they can be specified by passing arguments to a function. 

37) 
38 # For a slightly different approach to anagramming, 
39 #+ see the agram2.sh script. 


See also Example 29-4, Example 16-25, and Example A-9. 


Use "anonymous here documents" to comment out blocks of code, to save having to individually 
comment out each line with a #. See Example 19-11. 

e 
Running a script on a machine that relies on a command that might not be installed is dangerous. Use 
whatis to avoid potential problems with this. 


CMD=command1 # First choice. 
PlanB=command2 # Fallback option. 


command test-$(whatis "SCMD" | grep 'nothing appropriate") 
If 'commandl' not found on system , 'whatis' will return 
+ "commandl1: nothing appropriate." 


1 
2 
3 
4 
3 
6 
d 
8 A safer alternative is: 

9 command test-$(whereis "SCMD" | grep \/) 
0 But then the sense of the following test would have to be reversed, 
jl 

2 

3 

4 

E 

6 


+ since the $command test variable holds content only if 
al + the SCMD exists on the system. 
i (Thanks, bojster.) 
Hi 
le ir [I -z2 "Seeownune rest" Ji # Check whether command present. 
17 then 
18 SCMD optionl option2 # Run commandl with options. 
19 else # Otherwise, 
20 SPlanB #+ run command2. 
21 3 


An if-grep test may not return expected results in an error case, when text is output to stderr, rather 
that stdout. 


1 if ls -1 nonexistent filename | grep a 'No such file or directory' 
2 then echo “rile \Ynemexiscemc_tilenane\" does mot excu." 
S aal 


Redirecting stderr to stdout fixes this. 


if ls -l1 nonexistent filename 2>&1 | grep -q 'No such file or directory' 


i AAAA 


ib 
2 
3 chem echo “rile Wrnoneziletent ti lenane VU doss not esie" 
a dea 

E 

6 


W^ meus. Chris Narccin, kor JoxoEdLeKg Mle (GENE. 
* If you absolutely must access a subshell variable outside the subshell, here's a way to do it. 


TMPFILE-tmpfile # Create a temp file to store the variable. 


( # Inside the subshell 

inner variable-Inner 

echo $inner variable 

echo $inner variable >>STMPFILE # Append to temp file. 
) 


E 


=) ny (ap de (ys) Je») p 


8 

S # Outside the subshell 
10 
ii echo; Scho “4 Up. (exem) 
12 echo S$inner variable 4 Null, as expected. 
l3 echo M"----- "; echo 

14 

15 # Now 

16 read inner variable <STMPFILE # Read back shell variable. 
L7 sen =E "EUMD EU # Get rid of temp file. 

18 echo "$inner variable" # It's an ugly kludge, but it works. 


The run-parts command is handy for running a set of command scripts in a particular sequence, 


especially in combination with cron or at. 
e. 


For doing multiple revisions on a complex script, use the rcs Revision Control System package. 
Among other benefits of this is automatically updated ID header tags. The co command in rcs does a 
parameter replacement of certain reserved key words, for example, replacing # $Id$ ina script with 


something like: 


JL sx Sicle insillo=woriled.sin,w loi 2004/10/16 02:43:03 bozo mayo $ 


35.7.2. Widgets 


It would be nice to be able to invoke X-Windows widgets from a shell script. There happen to exist several 
packages that purport to do so, namely Xscript, Xmenu, and widtools. The first two of these no longer seem to 
be maintained. Fortunately, it is still possible to obtain widtools here. 


The widtools (widget tools) package requires the XForms library to be installed. Additionally, the 
Makefile needs some judicious editing before the package will build on a typical Linux system. Finally, 
three of the six widgets offered do not work (and, in fact, segfault). 


The dialog family of tools offers a method of calling "dialog" widgets from a shell script. The original dialog 
utility works in a text console, but its successors, gdialog, Xdialog, and kdialog use X-Windows-based widget 
sets. 


Example 35-20. Widgets invoked from a shell script 


! /bin/bash 
dialog.sh: Using 'gdialog' widgets. 


Must have 'gdialog' installed on your system to run this script. 
Or, you can replace all instance of 'gdialog' below with 'kdialog' 
Version 1.1 (corrected 04/05/05) 


This script was inspired by the following article. 
Seripcine For OC Prouet venen " Joy Mareo IP LOIeSIEIE iL, 
LINUX JOURNAL, Issue 113, September 2003, pp. 86-9. 
Thank you, all you good people at LJ. 


Input error in dialog box. 
E INPUT-65 

Dimensions of display, input widgets. 
EIGHT-50 


di 


=) ce», (Gr dex (93) fh [3 55) wer (ex c ep) (Oa oS (95 de» (5 


18 WIDTH-60 

1$ 

20 # Output file name (constructed out of script name). 

21 OUTFILE=$0.output 

22 

23 4$ Display this script in a text widget. 

24 gdialog --title "Displaying: $0" --textbox $0 SHEIGHT S$WIDTH 
25 

26 

AT 

28 # Now, we'll try saving input in a file. 

29 echo -n "VARIABLE-" > SOUTFILI 
30 ejeliieillec -ticle YUisiExe Imot importo Vimininsie vaciaole, 91lexeuseg Y \ 
31 SHEIGHT SWIDTH 2>> SOUTFILE 
32 

33 

34. ase [p Vegu e @ | 

29 c» iE Goce jexseWelbilers ro Cioci eiie Gracis. 


LES 


36 then 

S echo xccl ec Wdralog box wwe errors.” 

38 else 

39 echo Varro (Ss) aim \"challog [oos execuiciom . " 

40 Oi, Cllaclkecl row YWCeueeail”, istaa. or YOR ISa 
41 rm SOUTFILE 

42 exit SE_INPUT 


dis Eu 

44 

45 

46 

47 # Now, we'll retrieve and display the saved variable. 

AS s SOUTFILE # 'Source' the saved fil 

49 echo "The variable input in the \"input box\" was: "SVARIABLE"" 
50 

51. 

52 rm SOUTFILE # Clean up by removing the temp file. 

53 # Some applications may need to retain this file. 
54 

55 exit $? 

56 

57 4 Exercise: Rewrite this script using the 'zenity' widget set. 


The xmessage command is a simple method of popping up a message/query window. For example: 


1 xmessage Fatal error in script! -button exit 
The latest entry in the widget sweepstakes is zenity. This utility pops up GTK+ dialog widgets-and-windows, 
and it works very nicely within a script. 


iL ye unio (y 

2 i 

3 zenity --entry # Pops up query window 

4 #+ and prints user entry to stdout. 

5 

6 # Also try the --calendar and --scale options. 
7 p 

8 

9 answer-$( get info ) # Capture stdout in Sanswer variable. 
10 


11 echo "User entered: "Sanswer"" 
For other methods of scripting with widgets, try Tk or wish (Tcl derivatives), PerlTk (Perl with Tk 
extensions), tksh (ksh with Tk extensions), XForms4Perl (Perl with XForms extensions), Gtk-Perl (Perl with 
Gtk extensions), or PyQt (Python with Qt extensions). 


Va 
e 
>< 
et 


Prev Home 


Optimizations Up Security Issues 
Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 
Prev Chapter 35. Miscellany Next 


35.8. Security Issues 
35.8.1. Infected Shell Scripts 


A brief warning about script security is indicated. A shell script may contain a worm, trojan, or even a virus. 
For that reason, never run as root a script (or permit it to be inserted into the system startup scripts in 
/etc/rc.d)unless you have obtained said script from a trusted source or you have carefully analyzed it to 
make certain it does nothing harmful. 


Various researchers at Bell Labs and other sites, including M. Douglas McIlroy, Tom Duff, and Fred Cohen 
have investigated the implications of shell script viruses. They conclude that it is all too easy for even a 
novice, a "script kiddie," to write one. [1] 


Here is yet another reason to learn scripting. Being able to look at and understand scripts may protect your 
system from being compromised by a rogue script. 


35.8.2. Hiding Shell Script Source 


For security purposes, it may be necessary to render a script unreadable. If only there were a utility to create a 
stripped binary executable from a script. Francisco Rosales' shc -- generic shell script compiler does exactly 
that. 


Unfortunately, according to an article in the October, 2005 Linux Journal, the binary can, in at least some 
cases, be decrypted to recover the original script source. Still, this could be a useful method of keeping scripts 
secure from all but the most skilled hackers. 


35.8.3. Writing Secure Shell Scripts 


Dan Stromberg suggests the following guidelines for writing (relatively) secure shell scripts. 


* Don't put secret data in environment variables. 

* Don't pass secret data in an external command's arguments (pass them in via a pipe or redirection 
instead). 

e Set your $PATH carefully. Don't just trust whatever path you inherit from the caller if your script is 
running as root. In fact, whenever you use an environment variable inherited from the caller, think 
about what could happen if the caller put something misleading in the variable, e.g., if the caller set 


SHOME to /etc. 
Notes 


[1] See Marius van Oers' article, Unix Shell Scripting Malware, and also the Denning reference in the 


bibliography. 
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35.9. Portability Issues 


It is easier to port a shell than a shell script. 


--Larry Wall 
This book deals specifically with Bash scripting on a GNU/Linux system. All the same, users of sh and ksh 
will find much of value here. 


As it happens, many of the various shells and scripting languages seem to be converging toward the POSIX 
1003.2 standard. Invoking Bash with the -—posix option or inserting a set -o posix at the head of a script 
causes Bash to conform very closely to this standard. Another alternative is to use a Z//bin/sh sha-bang header 
in the script, rather than Z//bin/bash. [1] Note that /bin/shisalinkto /bin/bash in Linux and certain 
other flavors of UNIX, and a script invoked this way disables extended Bash functionality. 


Most Bash scripts will run as-is under ksh, and vice-versa, since Chet Ramey has been busily porting ksh 
features to the latest versions of Bash. 


On a commercial UNIX machine, scripts using GNU-specific features of standard commands may not work. 
This has become less of a problem in the last few years, as the GNU utilities have pretty much displaced their 
proprietary counterparts even on "big-iron" UNIX. Caldera's release of the source to many of the original 
UNIX utilities has accelerated the trend. 


Bash has certain features that the traditional Bourne shell lacks. Among these are: 


* Certain extended invocation options 

e Command substitution using $( ) notation 

* Brace expansion 

* Certain array operations, and associative arrays 
* The double brackets extended test construct 

* The double-parentheses arithmetic-evaluation construct 
e Certain string manipulation operations 

* Process substitution 

* A Regular Expression matching operator 

* Bash-specific builtins 

* Coprocesses 


See the Bash F.A.Q. for a complete listing. 


35.9.1. A Test Suite 


Let us illustrate some of the incompatibilities between Bash and the classic Bourne shell. Download and 
install the "Heirloom Bourne Shell" and run the following script, first using Bash, then the classic sh. 


Example 35-21. Test Suite 


#!/bin/bash 
# test-suite.sh 
# A partial Bash compatibility test suite. 


# Double brackets (test) 


1 
2 
3 
4 
E 
6 
7 String="Double brackets supported?" 


8 echo -n "Double brackets test: " 

9 if [[ "SString" = "Double brackets supported?" ]] 
10 then 

JL echo "PASS" 

12 else 

L3 echo "FAIL" 

l4 fi 

LS 

16 

17 # Double brackets and regex matching 
18 String="Regex matching supported?" 
19 echo -n "Regex matching: " 
AO) ax [D SPSSn =< deosos matching* ]] 
21 then 
22 echo "PASS" 
23 else 
24 echo "FAIL" 
25 it 
26 
AT 
28 # Arrays 


test_arr=FAIL 

Array-( If supports arrays will print PASS ) 
test arr-S$(Array[5]) 

echo "Array test: $test arr" 


# Completing this script is an exercise for the reader. 
# Add to the above similar tests for double parentheses, 
#+ brace expansion, $() command substitution, etc. 


CO CO CO CO CO CO CO CO CO CO Dd 
We) (eer =a) ny (Gn dex (Os) ik) TY e». (3e) 


exit $? 


Notes 


[1] Or, better yet, #!/bin/env sh. 
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35.10. Shell Scripting Under Windows 


Even users running that other OS can run UNIX-like shell scripts, and therefore benefit from many of the 
lessons of this book. The Cygwin package from Cygnus and the MKS utilities from Mortice Kern Associates 
add shell scripting capabilities to Windows. 


There have been intimations that a future release of Windows will contain Bash-like command-line scripting 
capabilities, but that remains to be seen. 
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Chapter 36. Bash, versions 2, 3, and 4 


36. 


1. Bash, version 2 


The current version of Bash, the one you have running on your machine, is most likely version 2.xx.yy, 
3.xx. yy, or 4.xx.yy. 


bash$ echo $BASH VERSION 


Sica 


25(1)-release 


The version 2 update of the classic Bash scripting language added array variables, string and parameter 
expansion, and a better method of indirect variable references, among other features. 


Example 36-1. String expansion 


Xo) es) | (ex (On dex (3 PS) [| (€» o (v9) Sa) (e»y (nn de Ge) |» [5 


#!/bin/bash 


# String expansion. 
# Introduced with version 2 of Bash. 


i? Serince ie the Torm S sere! 
#+ have the standard escaped characters interpreted. 


echo $'Ringing bell 3 times \a \a \a' 
# May only ring once with certain terminals. 
# Or 
# May not ring at all, depending on terminal settings. 
echo S Tares tonn eech NE NE yE. 
echo $'10 newlines \n\n\n\n\n\n\n\n\n\n' 
echo $'\102\141\163\150' 
# B a S h 
# Octal equivalent of characters. 


exit 


Example 36-2. Indirect variable references - the new way 


(Se) c @y Gal HG IS) [ey Wey ee 3] fo, Gn) des. (G3 IRS) [55 


19 
20 
Zakk 


#!/bin/bash 


# Indirect variable referencing. 
# This has a few of the attributes of references in C++. 


a-letter of alphabet 
letter of alphabet-z 


echo "a = $a" # Direct reference. 


eco. View a = {lary # Indirect reference. 
# The S{!variable} notation is more intuitive than the old 
#+ eval varl-N$$var2 


echo 


t-table cell 3 

table cell 3-224 

exeo Wie = Se # t = 24 
table_cell_3=387 


echo "Value of t changed to ${!t}" 


# 387 


# No 'eval' necessary. 

# This is useful for referencing members of an array or table, 
#+ or for simulating a multi-dimensional array. 

# An indexing option (analogous to pointer arithmetic) 

#+ would have been nice. Sigh. 

exit 0 

# See also, ind-ref.sh exampl 


Example 36-3. Simple database application, using indirect variable referencing 


NPRPRPPRP PPP PY 
COMIDTDHKRWNHHFOWOAIDGUBRWNE 


MNNNN 
OB WNE 
w UJ UJ UJ UJ 


No PN 
-] Oo 


CO CO CO CO CO CO CO CO CO CO DNO DN 
We) (99) «| «ow (On dE CS) IS) TS) 2 wep (99 


= e 
FONAS 


Pb am am Qm um aus 
(69) sal (ox (uL des Gor IS) 


#!/bin/bash 


# 
# 


Se HE 


UJ UJ UJ UJ UJ 


UJ) UJ UJ UJ UJ 


resistor-inventory.sh 


Simple database / table-lookup application. 


1724 value-1000 

1724 powerdissip-.25 

1724 colorcode-"brown-black-red" 
1724 loc-24N 

1724 inventory-243 


1725 value-10000 

1725 powerdissip-.125 

1725 colorcode-"brown-black-orange" 
1725 loc-24N 

1725 inventory-89 


# 
Data 
1723_value=470 # Ohms 
1723_powerdissip=.25 # Watts 
1723 colorcode-"yellow-violet-brown" # Color bands 
11729 Mex dE Ts. # Where they are 
1723 inventory-78 # How many 


echo 


PS3-'Enter catalog number: ' 


echo 


select catalog number in "B1723" "B1724" 


do 


Inv-$(catalog number) inventory 
Val-$(catalog number) value 

Pdissip=$ {catalog_number} powerdissip 
Loc=$ {catalog_number}_loc 

Ccode=$ {catalog_number }_colorcode 


echo 
echo "Catalog number Scatalog_number:" 


# Now, retrieve value, using indirect referencing. 


DRT 


echo "There are $(!Inv) of [S${!Val} ohm / ${!Pdissip} watt]\ 


^ 


resistors in stock." # 


^ 


echo "These are located in bin # ${!Loc}." 


49 Echa Wihieslie color coce as WU Ceroxeke LN sU 


50 

Sil break 

52 done 

53 

54 echo; echo 

55 

56 Exercises: 

5 Wd ———————-——- 

58 1) Rewrite this script to read its data from an external file. 

59 2) Rewrite this script to use arrays, 

60 #+ rather than indirect variable referencing. 

61 Which method is more straightforward and intuitive? 

62 Which method is easier to code? 

63 

64 

65 Notes: 

66 up ————- 

67 Shell scripts are inappropriate for anything except the most simple 
68 #+ database applications, and even then it involves workarounds and kludges. 
69 Much better is to use a language with native support for data structures, 
70 #+ such as C++ or Java (or even Perl). 

p 

72 exit 0 


Example 36-4. Using arrays and other miscellaneous trickery to deal four random hands from a deck of 
cards 


!/bin/bash 
cards.sh 


Deals four random hands from a deck of cards. 


UNPICKED-0 
PICKED-1 


DUPE CARD-99 


ER LIMIT-0 
UPPER LIMIT-51 
CARDS. IN SUIT-13 
CARDS-52 


declare -a Deck 

declare -a Suits 

declare -a Cards 

# It would have been easier to implement and more intuitive 
#+ with a single, 3-dimensional array. 


NPRPRPRPP PPP PY 
O (o 0» -1 O Oi i (). M P. O xo 0 «1 O O1 4 CQ Io E 
[e 
= 


21 # Perhaps a future version of Bash will support multidimensional arrays. 
22 

25 

2b sugli dee (0) 

25. 1 

26 i-S$LOWER LIMIT 

27 until [ "$i" -gt S$UPPER LIMIT | 

28 do 


Deck [i] =SUNPICKED # Set each card of "Deck" as unpicked. 
ige Ti = 10 

done 

echo 


) 


C0 C0 C0 CO WN 
ges (5 [n9 [5 «9» We 


S5 abastpsselabe Sales, (0) 
36 ( 

37 Swan Jpelee Clube 

38 Suits[1]=D #Diamonds 
39 Suits[2]=H #Hearts 
40 Suits[3]=S #Spades 
AD t 

42 

AS) alinalipitellabwex (Cyeueeks. (0) 
44 { 


AS (Qeuaole—(2. 9 4 5 6 7 e 9 10 J © KA 

46 # Alternate method of initializing an array. 

Ab Nt 

48 

ANS) Toug. cies i 

SO 4 

51 card number-$RANDOM 

52 ler "eel sowbhüloresc B= SCNNDS s Resriiee. cence o (Ü — Sil, shoes, 52 Carde, 


53 if [ "${Deck[card_number]}" -eq S$UNPICKED ] 
54 then 

55 Deck[card number]-$PICKED 
56 return $card number 

57 else 

58 return S$DUPE CARD 

59 Fi 

60 } 

61 

62 parse_card () 

63 4 


64 number=S1 

65 let "suit number = number / CARDS IN SUIT" 
66 suit=${Suits[suit_number] } 

67 exelmo =a VSeumieo 

68 let "card_no = number % CARDS_IN_SUIT" 

69 Card=${Cards[card_no] } 

WOES ds Cas 

"1 4 Print cards in neat columns. 


"2 9 

TS 

74 seed_random () # Seed random number generator. 

75 4 # What happens if you don't do this? 


76 seed= eval date +%s° 

77 let "seed %= 32766" 

78 RANDOM-$seed 

79 # What are some other methods 

80 #+ of seeding the random number generator? 


DUE 

82 

83 deal cards () 

oo 4 

85 echo 

86 

87 cards_picked=0 

88 while [ "Scards_picked" -le SUPPER_LIMIT ] 
89 do 

90 pick_a_card 

9L t-$? 

92 

93 if [ "St" -ne $DUPE CARD ] 

94 then 

95 parse card $t 

96 

97 u=Scards_picked+1 

98 # Change back to 1-based indexing (temporarily). Why? 
99) let "u %= SCARDS IN SUIT" 


100 ai [ Vu" eg © | # Nested if/then condition test. 


104 iE # Each hand set apart with a blank line. 


106 let "cards picked += 1" 
107 3E aL 
108 done 


eH 
n] o 
(c) We) 


echo 
return 0 


) 


# Structured programming: 
# Entire program logic modularized in functions. 


errererrrrr rr 


119 # 
120 seed_random 

121 initialize Deck 
1L22 munene LAS Suae 
orm eral nazer Cards 
124 deal cards 

125 # 


127 «exit 


131 Exercise 1: 
1132 Add comments to thoroughly document this script. 


134 Exercise 2: 
1.85 Add a routine (function) to print out each hand sorted in suits. 
136 You may add other bells and whistles if you like. 


138 Exercise 3: 
139 Simplify and streamline the logic of the script. 
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36.2. Bash, version 3 


On July 27, 2004, Chet Ramey released version 3 of Bash. This update fixed quite a number of bugs and 
added new features. 


Some of the more important added features: 


A new, more generalized {a..z} brace expansion operator. 


1 #!/bin/bash 

E 

3S 1tou aL sum 15,10] 

4 # Simpler and more straightforward than 

S sp deus cb aia (eec 10) 

6 do 

7 Seine m Wis W 

8 done 

9 

10 echo 

d 

12 c 1 2 954) s ee im 

13 

14 

L5 

le a? Or JUSt 

17 

LS emo (eoomj Ab Gide 3E pim 3. 3| JE Jb jp 3s) (Y [9 (p 3S Ss EUW i x Y? x 
9 echo {e..m} e 3E cpm sk p Je dL ie 
20 echo {z..a} “e\VeRWeUeES eG Pp Onmmlikiyjihng ted eoa 
21 Works backwards, too. 
22. Sen 125.3 5390] 25 26 27 28 29 30 
23 echo Moon A cu IU D 
24 seine 4X8] xo we ca qp eos ee er dox Het we! 
25 Shows (some of) the ASCII characters between Z and a, 
26 + but don't rely on this type of behavior because 
27 echo {]..a} ESSE 
28 Why? 
29 
30 
31 4 You can tack on prefixes and suffixes. 


w 
N 


echo “Number ASA A Tc 
# Number #1, Number #2, Number #3, Number #4, 


# You can concatenate brace-expansion sets. 

echo 42-9 )4xc. M ope Mom 

ls sp diy de dm a OZ d 2y F 2m a Se ow 3y T Se a 
Generates an algebraic expression. 

This could be used to find permutations. 


C0 CO CO CO CO CO CO 
Wey (per =a) (ex Gal des (o3 


# You can nest brace-expansion sets. 

BEING elc olio tio «Diy 

ajo e d 2 S 

The "comma operator" splices together strings. 


# Unfortunately, brace expansion does not lend itself to parameterization. 
varl=1 

var2=5 

echo. {Svar s Vas sw fil, oS} 


OP PF bP PEP BP SS 
© Wey (ey c] x Gal dE» (9 [S3 f^ «e» 


on 
[En 


e The ${!array[@]} operator, which expands to all the indices of a given array. 


1 #!/bin/bash 
2 
3 Array-(element-zero element-on lement-two element-three) 
4 
5 echo ${Array[0]} element-zero 
6 First element of array. 
7 
8 echo ${!Array[@]} O a 23 
9 All the indices of Array. 
10 
iil sow a 3um e Arcay ©] | 
IZ, (clo) 
13 echo ${Array[i] } element-zero 
14 element-one 
LS element-two 
16 element-three 
17 
LS All the elements in Array. 
19 done 
e 
The =~ Regular Expression matching operator within a double brackets test expression. (Perl has a 
similar operator.) 
1 #!/bin/bash 
2 
3 variable="This is a fine mess." 
4 
5 echo “Swawrialole 
6 
7 # Regex matching with =~ operator within [[ double brackets ]]. 
8 iu [I "Swesuxelle" =~ T..:2osoosc fin*es* ]] 
9 # NOTE: As of version 3.2 of Bash, expression to match no longer quoted. 
10 then 
LI echo "match found" 
12 # match found 
U3) sa 
Or, more usefully: 
1 #!/bin/bash 
2 
3 input-$1 
4 
5 
6 ae [p “Suimowe” == vo-9] r9) r9-9]-rre-9]/0-91-4[90-9] [9-9] /0-91 [0-9] 11 
7 # ^ NOTE: Quoting not necessary, as of version 3.2 of Bash. 
8 # NNN-NN-NNNN (where each N is a digit). 
9 then 
10 echo "Social Security number." 
LI # Process SSN. 
12 else 
1.3 echo "Not a Social Security number!" 
a # Or, ask for corrected input. 
US) ss 
For additional examples of using the =~ operator, see Example A-29, Example 19-14, Example A-35, 
and Example A-24. 
e 


The new set -o pipefail option is useful for debugging pipes. If this option is set, then the exit 
status of a pipe is the exit status of the last command in the pipe to fail (return a non-zero value), 
rather than the actual final command in the pipe. 


See Example 16-43. 


d The update to version 3 of Bash breaks a few scripts that worked under earlier versions. Test critical 
legacy scripts to make sure they still work! 


As it happens, a couple of the scripts in the Advanced Bash Scripting Guide had to be fixed up (see 
Example 9-4, for instance). 


36.2.1. Bash, version 3.1 


The version 3.1 update of Bash introduces a number of bugfixes and a few minor changes. 


e The += operator is now permitted in in places where previously only the = assignment operator was 


recognized. 
1 a-1 
2 echo $a # 1 
3 
4 at=5 # Won't work under versions of Bash earlier than 3.1. 
5 echo Sa # 15 
6 
7 at=Hello 
8 echo Sa # 15Hello 


Here, += functions as a string concatenation operator. Note that its behavior in this particular context 
is different than within a let construct. 


1 a-1 

2 echo Sa # 1 

3 

4 let a+=5 # Integer arithmetic, rather than string concatenation. 
5 echo $a # 6 

6 

7 let a+=Hello # Doesn't "add" anything to a. 

8 echo $a # 6 


36.2.2. Bash, version 3.2 


This is pretty much a bugfix update. 


* In global parameter substitutions, the pattern no longer anchors at the start of the string. 
e The -—wordexp option disables process substitution. 


e The =~ Regular Expression match operator no longer requires quoting of the pattern within [L... |]. 


In fact, quoting in this context is not advisable as it may cause regex evaluation to fail. 
Chet Ramey states in the Bash FAQ that quoting explicitly disables regex evaluation. 
See also the Ubuntu Bug List and Wikinerds on Bash syntax. 


Setting shopt -s compat31 in a script causes reversion to the original behavior. 
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36.3. Bash, version 4 


Chet Ramey announced Version 4 of Bash on the 20th of February, 2009. This release has a number of 
significant new features, as well as some important bugfixes. 


Among the new goodies: 


* Associative arrays. 


An associative array can be thought of as a set of two linked arrays -- one holding the data, and the 


other the keys that index the individual elements of the data array. 


Example 36-5. A simple address database 


#!/bin/bash4 
# fetch_address.sh 


declar A address 
# -A option declares associative array. 


address[Charles]="414 W. 10th Ave., Baltimore, MD 21236" 
address [John]="202 E. 3rd St., New York, NY 10009" 
address [Wilma]="1854 Vermont Ave, Los Angeles, CA 90023" 


echo "Charles's address is ${address[Charles]}." 

# Charles's address is 414 W. 10th Ave., Baltimore, MD 21236. 
echo "Wilma's address is ${address[Wilma]}." 

# Wilma's address is 1854 Vermont Ave, Los Angeles, CA 90023. 
echo "John's address is ${address[John]}." 

5; Jonass cexokeheexsis als 202 ma Sel Sits, New Yorke, IAG ILOWOS), 


e| ox, (Gal de Go) [sp [L2- Gy Wey resp | (ry Gal PSS (ee) [Sy I 


Example 36-6. A somewhat more elaborate address database 


!/bin/bash4 
fetch address-2.sh 
A more elaborate version of fetch address.sh. 


SUCCESS=0 
E_DB=99 # Error code for missing entry. 


declar A address 
-A option declares associative array. 


store address () 


( 
axelebcies e || Sd] $2 
return $? 


fetch address () 
( 


Sy Wer (es | (ex Gal ge (35 [m3 [L5 €» (er (G9? | (x, (On Es ah [5 [5 


N H 


21 
22 
23 
24 
25 
26 
27 
28 
AG 
30 
EI 
32 
Ss 
34 
S5 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 


* Enhancements to the case construct: the ; ; & and ; & terminators. 


ie [E 
then 
echo 


ws 


E 


ít 


(address[$1])" ] 


DB 


] 


"Sl's address is not in database." 
dee Du Ha 


echo VSil"s ackhikess ig Siackclieass [Si] }.” 


xem SP 


store_address 
store_address 
store_address 
Exercise: 


"Charles Jones" 
Wala Smee nh 192 
"Wilma Wilson" 


"414 W. 10th Ave., 
02 m, SEC Som 
"1854 Vermont Ave, 


Rewrite th 


fetch address 
C 
fetch_address 


fetch address 


fetch address 


exit $? 


# In this case, 


abov 


then assign field 1 to name, 


"Charles Jones" 


harles Jones's address is 414 W. 


"Wilma Wilson" 


Wilma Wilson's address is 1854 Vermont Ave, 


W rolon Smee nN 


John Smith's address is 202 


"Bozo Bozeman" 


Bozo Bozeman's address is not 


Example 36-7. Testing characters 


#!/bin/bash4 


O 


test char 


{ 


exit code 


10th Ave., 


jm. Seoce Siong 


in database. 


99, 


Baltimore, 


Los Angeles, 


Baltimore, 


Los Angeles, 


MID) ANS! 


New York, NY 10009" 


CA JOOS 


store_address calls to read data from a file, 
field 2 to address in the array. 
Each line in the file would have a format corresponding to the above. 


Use a while-read loop to read from file, sed or awk to parse the fields. 


MD 212365 
CA SOZ 


New York, NY 10009. 


Sines chet as PUMCELGN mec wien, 


case WSa qmm 
L| smseasees]! ) echo "Si is a psamuelle cimsuecter. "rp 
# The ;;& terminator continues to the next pattern test. 
LI Bedbewwagl] ) echo YEL is ain ceullerewAeweWureneahe Character, UP pi 
[ieadjomes i) ) echo “Sil as cum culjiu/oeuie gsusexcites. asp 
[[:lower:]] ) echo "$1 is a lowercase alphabetic character.";;& 
[fseagaies i ) echo VS us cum Marcile Gluersctexs." ps 
# The ;& terminator executes the next statement 
2; 2; 2; (a (8 (a (2 (d ) echo LE Sia kas Eat oru or edu SA5 AS EAT SAS TAS dod SAE AS EAT S OS odi sucus 
# EON Oa even with a dummy pattern. 
esac 
} 
echo 


test char 3 


# 3 is a printable character. 
# 3 is an alpha/numeric character. 


# 3 is an numeric character. 
# KKK KK KKK KEK KKK KKK KKK KKK KKKKKKK KKK 


echo 


test_char m 


# m is a printable character. 


29 # m is an alpha/numeric character. 

30 # m is an alphabetic character. 

31 # m is a lowercase alphabetic character. 
32 echo 

33 

3A iesu Ghar / 

35 a7 / cs a jJoseabeucelolke: Characters 

36 

37 echo 

38 

39 # The ;;& terminator can save complex if/then conditions. 
40 # The ;& is somewhat less useful. 


e The new coproc builtin enables two parallel processes to communicate and interact. As Chet Ramey 
states in the Bash FAO [1], ver. 4.01: 


There is a new 'coproc' reserved word that specifies a coprocess: 

an asynchronous command run with two pipes connected to the creating 
shell. Coprocs can be named. The input and output file descriptors 

and the PID of the coprocess are available to the calling shell in 
variables with coproc-specific names. 


George Dimitriu explains, 

"... coproc ... is a feature used in Bash process substitution, 

which now is made publicly available." 

This means it can be explicitly invoked in a script, rather than 

just being a behind-the-scenes mechanism used by Bash. 

See http://linux010.blogspot.com/2008/12/bash-process-substitution.html. 


Coprocesses use file descriptors. File descriptors enable processes and pipes to communicate. 


1 #!/bin/bash4 

2 # A coprocess communicates with a while-read loop. 

3 

4 

5 COPO | Caw me cece cork Sleep 2p } 

6 d dS 

7 4 Try running this without "sleep 2" and see what happens. 

8 

9 while read -u ${COPROC[0]} line $  S(COPROC[0]) is the 

10 do #+ file descriptor of the coprocess. 
dil echo "Sline" | sed -e 's/line/NOT-ORIGINAL-TEXT/' 

12 done 

13 

14 kill SCOPROC PID # No longer need the coprocess, 
LS ier SO kai aves ID), 


But, be careful! 


1 #!/bin/bash4 

2 

3 echo; echo 

4 a=aaa 

5 b=bbb 

6 ecce 

7 

8 coproc echo "one two three" 

9 while read -u $(COPROC[0]) a b c; # Note that this loop 
10 do #+ runs in a subshell. 


JL echo "Inside while-read loop: "; 
LZ eco "a = Sale echo Yo = $o: echo “G = Sg" 
13 ecne "coproc cile Ceccriprcors so COPROG LOI}! 


14 done 

5) 

16 4$ a = one 

17 # b = two 

18 # c = three 

1S $0 SO iex, SO Goocl, lou 
20 

21l elg W Y 


22 echo "Outside while-read loop: " 

23 echo "a = $a" #a= 

24 echo "b = $b" #b = 

25 echo "c = $c" #c= 

26 echo "coproc file descriptor: ${COPROC[0]}" 
27 echo 

28 # The coproc is still running, but 

29 #+ it still doesn't enable the parent process 
30 #+ to "inherit" variables from the child process, the while-read loop. 
Sil 

32 # Compare this to the "badread.sh" script. 


D The coprocess is asynchronous, and this might cause a problem. It may terminate before 
another process has finished communicating with it. 


#!/bin/bash4 


coore Conane 4 Eor a im (0-10; clo echo "ime = Sip clomep } 
d A9 Wins Se m Umen" CG[BEOOSUS. 

read -u ${cpname[0] } 

echo SREPLY # index = 0 

echo ${COPROC[0] } #+ No output ... the coprocess timed out 


# after the first loop iteration. 


# However, George Dimitriu has a partial fix. 


(Clojsuerore! (ejshevebuls: f cor ab im HO, iO} Pp Cle echo Vinde = Salo cone? pileeys lz 
echo Inu 2 myo Get = SS wwe) } 
d MASS "ame Je m manec Cor roces. 


echo "I am main"$'\04' >&${cpname[1] } 
my fd=${cpname [0] } 
echo myfd-$myfd 


PRPRPPPRPRPRPHPE 
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### while read -u $myfd 
### do 

LEX echo SREPLY; 

### done 


NNN 
ow C) 


echo $cpname PID 


# Run this with and without the commented-out while-loop, and it is 
#+ apparent that each process, the executing shell and the coprocess, 
31 #+ waits for the other to finish writing in its own writ nabled pip 


WNNN h2 
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* The new mapfile builtin makes it possible to load an array with the contents of a text file without 
using a loop or command substitution. 


#!/bin/bash4 


mapfile Arrl < $0 
T Sane result as Arrl-( $(cat $0) ) 
echo "S(Arr1[80])" # Copies this entire script out to stdout. 


OSG SS) 


6 

y echo "--"s. echo 

8 

9 # But, not the same as read -a ay 


1 reacli -a Arra < 0 
11 echo "S${Arr2[@]}" # Reads only first line of script into the array. 
12 
13) Qeie 
e The read builtin got a minor facelift. The -t timeout option now accepts (decimal) fractional values 
[2] and the -i option permits preloading the edit buffer. [3] Unfortunately, these enhancements are 
still a work in progress and not (yet) usable in scripts. 
* Parameter substitution gets case-modification operators. 


1 #!/bin/bash4 

2 

3 var-veryMixedUpVariable 

4 echo S{var} veryMixedUpVariable 

5 echo Si{var~} VeryMixedUpVariable 

6 d a First char => uppercase. 
7 echo p vare) VERYMIXEDUPVARIABLE 

8 4 Wi All chars  --» uppercase. 
9 echo S{var, } veryMixedUpVariable 
10 4 a First char --> lowercase. 
iil echo varrr) verymixedupvariable 
12 # s All chars --> lowercase. 


The declare builtin now accepts the -1 lowercase and -c capitalize options. 


1 #!/bin/bash4 

2 

3 declare -1l vari Will change to lowercase 
4 varl-MixedCaseVARIABLE 

5 Cho Saul mixedcasevariable 

6 # Same effect as echo Svarl | tr A-Z a-z 
7 

8 declare -c var2 Changes only initial char to uppercase. 
9 var2-originally lowercase 

10 scho "Suwer2 Originally lowercase 

11 # NOT the same effect as echo $var2 | tr a-z A-Z 


* Brace expansion has more options. 
Increment/decrement, specified in the final term within braces. 


!/bin/bash4 


echo (40..60..2) 
40 42 44 46 48 50 52 54 56 58 60 
Ald th ven numbers, between 40 and 60. 


echo (60..40..2) 

60 58 56 54 52 50 48 46 44 42 40 

ALTI th ven numbers, between 40 and 60, counting backwards. 
In effect, a decrement. 

Seine (60> “AO. oT 

The same output. The minus sign is not necessary. 


But, what about letters and symbols? 
echo {X..d} 
ENE We (p.  * eec Toy del 19! 
echo (X..0.<.2) 
X zm o ^ " Ip el 
It seems to work for upper/lowercase letters, 
+ but the increment is a bit inconsistent on symbols. 


NPRPRPRP PPP Pee 
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Zero-padding, specified in the first term within braces, prefixes each term in the output with the same 
number of zeroes. 


bash4$ echo {010..15} 
(iO OL 092 OLS O14 Oils 


bash4S echo (000..10) 


000 001 002 003 004 005 006 007 008 009 010 


Substring extraction on positional parameters now starts with $0 as the zero-index. (This corrects an 
inconsistency in the treatment of positional parameters.) 


! /bin/bash4 
show-params.bash4 


Invoke this scripts with at least one positional parameter. 


E BADPARAMS-99 


ne [ —z Si" | 

then 
echo "Usage $0 paraml ..." 
exit $E BADPARAMS 

dE ab 


e 


e 


echo ${@:0} 


nep! 
OY O' 4& Q). I9. IB. O (o O0 -1 O Or i8 0) IN S 


# bash3 show-params.bash4 one two three 
# one two three 


bash4 show-params.bash4 one two three 
# show-params.bash4 one two three 


Nere 
© w e 4 
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22 # $0 Sl 82 eg 
e The new ** globbing operator matches filenames and directories recursively. 


1 #!/bin/bash4 
2 4 filelist.bash4 
3 
4 shopt -s globstar # Must enable globstar, otherwise ** doesn't work. 
5 # The globstar shell option is new to version 4 of Bash. 
6 
7 echo "Using *"; echo 
6 for Filename im = 
9 co 
LO echo "Sfilename" 
11 done # Lists only files in current directory (S$PWD). 
L2 
13 echo; cho " die cho 
14 
lS exeo '"ijgwey 9998) 
1G for filename ain =% 
JL do 
i18 echo "$filename" 
19 done # Lists complete file tree, recursively. 
20 
ZL (ele 
22 
23 Usine = 
24 
25 allmyfiles 
26 filelist.bash4 


27 


28 

29 

39 Weaning, € 

Sl 

32 allmyfiles 

33 allmyfiles/file.index.txt 

34 allmyfiles/my music 

35 allmyfiles/my music/me-singing-60s-folksongs.ogg 
36 allmyfiles/my music/me-singing-opera.ogg 

37 allmyfiles/my music/piano-lesson.1.0gg 

38 allmyfiles/my pictures 

39 allmyfiles/my pictures/at-beach-with-Jade.png 
40 allmyfiles/my pictures/picnic-with-Melissa.png 


AZUL e hste Basha 
e The new $BASHPID internal variable. 


There is a new builtin error-handling function named command_not_found_handle. 


1 #!/bin/bash4 

2 

3 command not found handle () 

4 ( # Accepts implicit parameters. 

E echo Vihe following comman! is moi wallscle WU SDN UU 

6 echo mith the folleming argument (e)s PUSA VuGSSymum 5 Sa. S5 
7 } s Sil, SZ, cue. are not esyollucwicihy passed to che irwiscienem. 

8 


9 bad command argl arg2 


10 
11 # The following command is not valid: "bad command" 
12 # With the following argument (s): "argi" "arg2" 


Editorial comment 


Associative arrays? Coprocesses? Whatever happened to the lean and mean Bash we have come to know 
and love? Could it be suffering from (horrors!) "feature creep"? Or perhaps even Korn shell envy? 


Note to Chet Ramey: Please add only essential features in future Bash releases -- perhaps for-each loops and 
support for multi-dimensional arrays. [4] Most Bash users won't need, won't use, and likely won't greatly 
appreciate complex "features" like built-in debuggers, Perl interfaces, and bolt-on rocket boosters. 


Notes 


[1] Copyright 1995-2009 by Chester Ramey. 
[2] This only works with pipes and certain other special files. 
[3] Butonly in conjunction with readline, i.e., from the command-line. 


[4] And while you're at it, consider fixing the notorious piped read problem. 
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Chapter 37. Endnotes 


37.1. Author's Note 


doce ut discas 


(Teach, that you yourself may learn.) 
How did I come to write a Bash scripting book? It's a strange tale. It seems that a few years back I needed to 
learn shell scripting -- and what better way to do that than to read a good book on the subject? I was looking 
to buy a tutorial and reference covering all aspects of the subject. I was looking for a book that would take 
difficult concepts, turn them inside out, and explain them in excruciating detail, with well-commented 
examples. [1] In fact, I was looking for this very book, or something very much like it. Unfortunately, it didn't 
exist, and if I wanted it, I'd have to write it. And so, here we are, folks. 


That reminds me of the apocryphal story about a mad professor. Crazy as a loon, the fellow was. At the sight 
of a book, any book -- at the library, at a bookstore, anywhere -- he would become totally obsessed with the 
idea that he could have written it, should have written it -- and done a better job of it to boot. He would 
thereupon rush home and proceed to do just that, write a book with the very same title. When he died some 
years later, he allegedly had several thousand books to his credit, probably putting even Asimov to shame. 
The books might not have been any good, who knows, but does that really matter? Here's a fellow who lived 
his dream, even if he was obsessed by it, driven by it . . . and somehow I can't help admiring the old coot. 


Notes 
[1] Thisis the notorious flog it to death technique. 
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37.2. About the Author 


Who is this guy anyhow? 


The author claims no credentials or special qualifications, [1] other than a compulsion to write. [2] This book 
is somewhat of a departure from his other major work, HOW-2 Meet Women: The Shy Man's Guide to 
Relationships. He has also written the Software-Building HOWTO. Of late, he has been trying his (heavy) 
hand at short fiction. 


A Linux user since 1995 (Slackware 2.2, kernel 1.2.1), the author has emitted a few software truffles, 
including the cruft one-time pad encryption utility, the mcalc mortgage calculator, the judge Scrabble 
adjudicator, the yawl word gaming list package, and the Quacky anagramming gaming package. He got off to 
a rather shaky start in the computer game -- programming FORTRAN IV on a CDC 3800 -- and is not the 
least bit nostalgic for those days. 


Living in a secluded desert community with wife and orange tabby, he cherishes human frailty, especially his 
own. [3] 


Notes 


[1] In fact, he is a school dropout and has no formal credentials or qualifications whatsoever. Aside from 
the ABS Guide, his main claim to fame is a First Place in the sack race at the Colfax Elementary School 
Field Day in June, 1958. 


[2] Those who can, do. Those who can't... getan MCSE. 
BI 


Sometimes it seems as if he has spent his entire life flouting conventional wisdom and defying the 
sonorous Voice of Authority: "Hey, you can't do that!" 
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37.3. Where to Go For Help 


The author will sometimes, if not too busy (and in a good mood), answer general scripting questions. [1] 
However, if you have a problem getting a specific script to work, you would be well advised to post to the 


comp.os.unix.shell Usenet newsgroup. 


If you need assistance with a schoolwork assignment, read the pertinent sections of this and other reference 
works. Do your best to solve the problem using your own wits and resources. Kindly do not waste the author's 
time. You will get neither help nor sympathy. [2] 


... Sophisticated in mechanism but possibly agile 
operating under noises being extremely 
suppressed ... 


--CI-300 printer manual 
Notes 


[1] E-mails from certain spam-infested TLDs (61, 202, 211, 218, 220, etc.) will be trapped by spam filters 
and deleted unread. If your ISP is located on one of these, please use a Webmail account to contact the 
author. 


[2] Well, if you absolutely insist, you can try modifying Example A-44 to suit your purposes. 
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37.4. Tools Used to Produce This Book 
37.4.1. Hardware 


A used IBM Thinkpad, model 760XL laptop (P166, 104 meg RAM) running Red Hat 7.1/7.3. Sure, it's slow 
and has a funky keyboard, but it beats the heck out of a No. 2 pencil and a Big Chief tablet. 


Update: upgraded to a 770Z Thinkpad (P2-366, 192 meg RAM) running FC3. Anyone feel like donating a 
later-model laptop to a starving writer <g>? 


Update: upgraded to a A31 Thinkpad (P4-1.6, 512 meg RAM) running FC8. No longer starving, and no 
longer soliciting donations «g». 


37.4.2. Software and Printware 


i. Bram Moolenaar's powerful SGML-aware vim text editor. 
ii. OpenJade, a DSSSL rendering engine for converting SGML documents into other formats. 
iii. Norman Walsh's DSSSL stylesheets. 
iv. DocBook, The Definitive Guide, by Norman Walsh and Leonard Muellner (O'Reilly, ISBN 
1-56592-580-7). This is still the standard reference for anyone attempting to write a document in 
Docbook SGML format. 
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37.5. Credits 


Community participation made this project possible. 'The author gratefully acknowledges that writing this 
book would have been unthinkable without help and feedback from all you people out there. 


Philippe Martin translated the first version (0.1) of this document into DocBook/SGML. While not on the job 
at a small French company as a software developer, he enjoys working on GNU/Linux documentation and 
software, reading literature, playing music, and, for his peace of mind, making merry with friends. You may 
run across him somewhere in France or in the Basque Country, or you can email him at feloy @free.fr. 


Philippe Martin also pointed out that positional parameters past $9 are possible using (bracket) notation. (See 
Example 4-5). 


Stéphane Chazelas sent a long list of corrections, additions, and example scripts. More than a contributor, he 
had, in effect, for a while taken on the role of co-editor for this document. Merci beaucoup! 


Paulo Marcel Coelho Aragao offered many corrections, both major and minor, and contributed quite a number 
of helpful suggestions. 


I would like to especially thank Patrick Callahan, Mike Novak, and Pal Domokos for catching bugs, pointing 
out ambiguities, and for suggesting clarifications and changes in the preliminary version (0.1) of this 
document. Their lively discussion of shell scripting and general documentation issues inspired me to try to 


make this document more readable. 


I'm grateful to Jim Van Zandt for pointing out errors and omissions in version 0.2 of this document. He also 
contributed an instructive example script. 


Many thanks to Jordi Sanfeliu for giving permission to use his fine tree script (Example A-16), and to Rick 
Boivie for revising it. 


Likewise, thanks to Michel Charpentier for permission to use his dc factoring script (Example 16-52). 
Kudos to Noah Friedman for permission to use his string function script (Example A-18). 


Emmanuel Rouat suggested corrections and additions on command substitution and aliases. He also 
contributed a very nice sample . bashrc file (Appendix L). 


Heiner Steven kindly gave permission to use his base conversion script, Example 16-48. He also made a 
number of corrections and many helpful suggestions. Special thanks. 


Rick Boivie contributed the delightfully recursive pb.sh script (Example 35-9), revised the tree.sh script 
(Example A-16), and suggested performance improvements for the monthlypmt.sh script (Example 16-47). 


Florian Wisser enlightened me on some of the fine points of testing strings (see Example 7-6), and on other 
matters. 


Oleg Philon sent suggestions concerning cut and pidof. 


Michael Zick extended the empty array example to demonstrate some surprising array properties. He also 
contributed the isspammer scripts (Example 16-41 and Example A-28). 


Marc-Jano Knopp sent corrections and clarifications on DOS batch files. 


Hyun Jin Cha found several typos in the document in the process of doing a Korean translation. Thanks for 
pointing these out. 


Andreas Abraham sent in a long list of typographical errors and other corrections. Special thanks! 


Others contributing scripts, making helpful suggestions, and pointing out errors were Gabor Kiss, Leopold 
Toetsch, Peter Tillier, Marcus Berglof, Tony Richardson, Nick Drage (script ideas!), Rich Bartell, Jess 
Thrysoee, Adam Lazur, Bram Moolenaar, Baris Cicek, Greg Keraunen, Keith Matthews, Sandro Magi, Albert 
Reiner, Dim Segebart, Rory Winston, Lee Bigelow, Wayne Pollock, "jipe," "bojster," "nyal," "Hobbit," 
"Ender," "Little Monster" (Alexis), "Mark," "Patsie," Peggy Russell, Emilio Conti, Ian. D. Allen, Hans-Joerg 
Diers, Arun Giridhar, Dennis Leeuw, Dan Jacobson, Aurelio Marinho Jargas, Edward Scholtz, Jean Helou, 
Chris Martin, Lee Maschmeyer, Bruno Haible, Wilbert Berendsen, Sebastien Godard, Bjón Eriksson, John 
MacDonald, John Lange, Joshua Tschida, Troy Engel, Manfred Schwarb, Amit Singh, Bill Gradwohl, E. 
Choroba, David Lombard, Jason Parker, Steve Parker, Bruce W. Clare, William Park, Vernia Damiano, Mihai 
Maties, Mark Alexander, Jeremy Impson, Ken Fuchs, Jared Martin, Frank Wang, Sylvain Fourmanoit, 
Matthew Sage, Matthew Walker, Kenny Stauffer, Filip Moritz, Andrzej Stefanski, Daniel Albers, Stefano 
Palmeri, Nils Radtke, Serghey Rodin, Jeroen Domburg, Alfredo Pironti, Phil Braham, Bruno de Oliveira 
Schneider, Stefano Falsetto, Chris Morgan, Walter Dnes, Linc Fessenden, Michael Iatrou, Pharis Monalo, 
Jesse Gough, Fabian Kreutz, Mark Norman, Harald Koenig, Dan Stromberg, Peter Knowles, Francisco Lobo, 
Mariusz Gniazdowski, Sebastian Arming, Chetankumar Phulpagare, Benno Schulenberg, Tedman Eng, 
Jochen DeSmet, Juan Nicolas Ruiz, Oliver Beckstein, Achmed Darwish, Dotan Barak, Richard Neill, Albert 
Siersema, Omair Eshkenazi, Geoff Lee, JuanJo Ciarlante, Cliff Bamford, Nathan Coulter, Ramses Rodriguez 
Martinez, George Dimitriu, Antonio Macchi, Tomas Pospisek, Andreas Kühne, Pádraig Brady, and David 
Lawyer (himself an author of four HOWTOs). 


My gratitude to Chet Ramey and Brian Fox for writing Bash, and building into it elegant and powerful 
scripting capabilities rivaling those of ksh. 


Very special thanks to the hard-working volunteers at the Linux Documentation Project. The LDP hosts a 
repository of Linux knowledge and lore, and has, to a great extent, enabled the publication of this book. 


Thanks and appreciation to IBM, Red Hat, the Free Software Foundation, and all the good people fighting the 
good fight to keep Open Source software free and open. 


Belated thanks to my fourth grade teacher, Miss Spencer, for emotional support and for convincing me that 
maybe, just maybe I wasn't a total loss. 


Thanks most of all to my wife, Anita, for her encouragement, inspiration, and emotional support. 
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37.6. Disclaimer 


(This is a variant of the standard LDP disclaimer.) 


No liability for the contents of this document can be accepted. Use the concepts, examples and information at 
your own risk. There may be errors, omissions, and inaccuracies that could cause you to lose data or harm 
your system, so proceed with appropriate caution. The author takes no responsibility for any damages, 
incidental or otherwise. 


As it happens, it is highly unlikely that either you or your system will suffer ill effects. In fact, the raison 
d'etre of this book is to enable its readers to analyze shell scripts and determine whether they have 


unanticipated consequences. 
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"Shelltris" and "shellitaire" at Shell Script Games. 


Mark Komarinski's Printing-Usage HOWTO. 


The Linux USB subsystem (helpful in writing scripts affecting USB peripherals). 


There is some nice material on I/O redirection in chapter 10 of the textutils documentation at the University 
of Alberta site. 


Rick Hohensee has written the osimpa 1386 assembler entirely as Bash scripts. 


Aurelio Marinho Jargas has written a Regular expression wizard. He has also written an informative book on 
Regular Expressions, in Portuguese. 


Ben Tomkins has created the Bash Navigator directory management tool. 


William Park has been working on a project to incorporate certain Awk and Python features into Bash. Among 
these is a gdbm interface. He has released bashdiff on Freshmeat.net. He has an article in the November, 2004 
issue of the Linux Gazette on adding string functions to Bash, with a followup article in the December issue, 
and yet another in the January, 2005 issue. 


Peter Knowles has written an elaborate Bash script that generates a book list on the Sony Librie e-book 
reader. This useful tool facilitates loading non-DRM user content on the Librie (and the newer 
PRS-50X-series devices). 


Tim Waugh's xmlto is an elaborate Bash script for converting Docbook XML documents to other formats. 


Philip Patterson's logforbash logging/debugging script. 


Of historical interest are Colin Needham's original International Movie Database (IMDB) reader polling 


scripts, which nicely illustrate the use of awk for string parsing. Unfortunately, the URL link is broken. 


Fritz Mehner has written a bash-support plugin for the vim text editor. He has also also come up with his own 
stylesheet for Bash. Compare it with the ABS Guide Unofficial Stylesheet. 


Penguin Pete has quite a number of shell scripting tips and hints on his superb site. Highly recommended. 


The excellent Bash Reference Manual, by Chet Ramey and Brian Fox, distributed as part of the bash-2-doc 
package (available as an rpm). See especially the instructive example scripts in this package. 


John Lion's classic, A Commentary on the Sixth Edition UNIX Operating System. 


The comp.os.unix.shell newsgroup. 


The dd thread on Linux Questions. 


The comp.os.unix.shell FAQ. 


Assorted comp.os.unix FAQs. 


The Wikipedia article covering dc. 


The manpages for bash and bash2, date, expect, expr, find, grep, gzip, In, patch, tar, tr, bc, xargs. The 
texinfo documentation on bash, dd, m4, gawk, and sed. 


Notes 


[1] It was hard to resist the obvious pun. No slight intended, since the book is a pretty decent introduction 
to the basic concepts of shell scripting. 
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Appendix A. Contributed Scripts 


These scripts, while not fitting into the text of this document, do illustrate some interesting shell programming 


techniques. They are useful, too. Have fun analyzing and running them. 


Example A-1. mailformat: Formatting an e-mail message 


!/bin/bash 
mail—format.sh (ver. 1.1): Format mail messages. 


Gets rid of carets, tabs, and also folds excessively long lines. 


Standard Check for Script Argument (s) 


ARGS-1 

E BADARGS-65 

E NOFILE-66 

if [ $4 -ne SARGS ] # Correct number of arguments passed to script? 
then 


echo "Usage: ~basename $0? filename" 
exit $E BADARGS 


Mop p opp ppppÀ:! 
O (o 0» -1 O Oi i$ (0. N HP O xo 0 -1 O O1 4 CQ I E 


# Delete carets and tabs at beginning of lines, 
#+ then fold lines to $MAXWIDTH characters. 
sed "Ssedscript" $1 | fold -s --width-$MAXWIDTH 
u =S Grau ce dede" 
#+ breaks lines at whitespace, if possible. 


i3 
ane dp esie. VSL) # Check abi fiile exists. 
then 
file name-$1 
21 else 
22 echo "File \"S1\" does not exist." 
BS exit SE_NOFILE 
24 fi 
DENM 
26 
27 MAXWIDTH=70 # Width to fold excessively long lines to. 
28 
29 Ye 
30 # A variable can hold a sed script. 
31 sedscript-'s/^»// 
32 s/^ "m/f 
Sich tay tau) 
34 ef m TEM 
35 # 
36 
37 
38 
39 


A e 
aY 


# This script was inspired by an article in a well-known trade journal 
#+ extolling a 164K MS Windows utility with similar functionality. 


# An nice set of text processing utilities and an efficient 
#+ scripting language provide an alternative to bloated executables. 


Cn um am am d» ds d» RA 
GS) Vor (es) SS] (ex; (On Wes ey TS) 
+ 


exit 


Example A-2. rn: A simple-minded file renaming utility 


This script is a modification of Example 16-22. 


#! /bin/bash 
# rn.sh 


# Very simpleminded filename "rename" utility (based on "lowercase.sh"). 
# 

# The "ren" utility, by Vladimir Lanin (lanin@csd2.nyu.edu), 

#+ does a much better job of this. 


ARGS=2 
E_BADARGS=85 
ONE=1 # For getting singular/plural right (see below). 


if [ $4 -ne "SARGS" ] 

then 
echo "Usage: 'basename $0! old-pattern new-pattern" 
# As in "rn gif jpg", which renames all gif files in working directory to jpg. 
exit $E BADARGS 

ical 


NPRPRPP PPP PP 
O (o 0» -1 O Ui i (). M HP. O (o 0 -1 O O1 i CQ I E 


number=0 # Keeps track of how many files actually renamed. 


NNN PN 
dex (8) [SS qp 


for filename in *$1* #Traverse all matching files in directory. 
do 
he SS Vemi lenene | s lt ssec Sm etc ee 
then 
fname-'basename $filename” iy ISXEJESQS OI path. 
n= echo Simenm | mee -e Ys/S1/s2/0~ 4 Substitute new for old in filename. 
mv Sfname $n # Rename. 
let "number += 1" 
ita 
done 


NNN 
| Oo) (Gal 


N 
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if [ "Snumber" -eq "SONE 
then 

echo "$number file renamed." 
else 

echo "Snumber files renamed." 
£d 


# For correct grammar. 


CO CO CO CO CO CO CO CO CO CO Dd 
We) Koo) SS) OY (Gal HS (Oe) [oy [SS Ke) CO 


A A 
= & 


exit $? 


Exercises: 


# What types of files will this not work on? 
# How can this be fixed? 


w SS dS uS uS a 
0 «p wy (rl des (03) [S3 
+ 


Example A-3. blank-rename: Renames filenames containing blanks 
This is an even simpler-minded version of previous script. 


! /bin/bash 


# 
# blank-rename.sh 
# 
# 


Substitutes underscores for blanks in all the filenames in a directory. 


o (Wal de (93 [S9 I 


ONE-1 # For getting singular/plural right (see below). 


7 number=0 # Keeps track of how many files actually renamed. 
8 FOUND=0 # Successful return value. 

9 

10 for filename in * #Traverse all files in directory. 

iil lo 

42 echo "S$Sfilename" | grep -q " " # Check whether filenam 

T3 aie [| S2 -eer SiO T #+ contains space(s). 

14 then 

LS fname=S filename # Yes, this filename needs work. 
16 ims Clove) Sinema | eo -e Vaf A Jg # Substitute underscore for blank. 
47 Invauds mc MES # Do the actual renaming. 

18 let "number += 1" 

19 Henn 

20 done 

21 

22 ie [ "Sumo" exer “SONY? T # For correct grammar. 

23 then 

24 echo "$number file renamed." 

25 else 

26 echo "$number files renamed." 

2.7 ie aL 

28 

29 exit 0 


Example A-4. encryptedpw: Uploading to an ftp site, using a locally encrypted password 


!/bin/bash 


Example "ex72.sh" modified to use encrypted password. 


Note that this is still rather insecure, 
+ since the decrypted password is sent in the clear. 
Use something like "ssh" if this is a concern. 


E_BADARGS=85 


aba [| =a; VESEY = 

then 
echo "Usage: 'basename $0` filename" 
exit SE BADARGS 

ipa 


Username=bozo # Change to suit. 
pword=/home/bozo/secret/password_encrypted. fil 
# File containing encrypted password. 


NPRPRPPRP PPP PY 
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21 Filename-' basename $1' # Strips pathname out of file name. 
22 

23 Server="XXX" 

24 Directory-"YYY" # Change above to actual server name & directory. 
215 

26 

27 Password= cruft «S$pword' # Decrypt password. 

28 Uses the author's own "cruft" file encryption package, 
29 #+ based on the classic "onetime pad" algorithm, 

30 #+ and obtainable from: 

31 #+ Primary-site: ftp: //ibiblio.org/pub/Linux/utils/file 
da ee Gruncc—0 s Z tee ei [Lilet] 

33 

34 


35 ftp -n $Server ««End-Of-Session 
36 user $Username $Password 


37 binary 

38 bell 

39 cd SDirectory 
40 put $Filename 


41 bye 

42 End-Of-Session 

43 =i Opelo iE. Vio" chiseloles: WTO Jogos 

44 Note that "bell" rings 'bell' after each file transfer. 
45 

46 exit 0 


Example A-5. copy-cd: Copying a data CD 


#!/bin/bash 
# copy-cd.sh: copying a data CD 


CDROM-/dev/cdrom # CD ROM device 
OF-/home/bozo/projects/cdimage.iso # output file 

# / XXXX/ XXXXXXXX/ Change to suit your system. 
BLOCKSIZE-2048 

# SPEED-10 # If unspecified, uses max spd. 
# DEVICE=/dev/cdrom older version. 

DEV ACH— Uses OON 


echo; echo "Insert source CD, but do *not* mount it." 
echo "Press ENTER when ready. " 
read ready # Wait for input, Sready not used. 


echo; echo "Copying the source CD to SOF." 
echo "This may take a while. Please be patient." 


dd if=SCDROM of=SOF bs-S$BLOCKSIZ] 


[za 


# Raw device copy. 


NPRPRPPRP PPP PY 
O (o 00 -1 O Oi i5 (). M PB. O «(o 0 -1 O O1 i CQ Io E 


24 

22 echo; cho "Remove data CD." 

As) «exea. \ iinsesce okere (CIDR. M 

24 echo "Press ENTER when ready. " 

25 read ready # Wait for input, Sready not used. 
26 

27 ecko WCejoyyiune; SO i) CDR, Y 


N 
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# cdrecord -v -isosize speed=SSPEED dev=SDEVICE SOF # Old version. 
wodim -v -isosize dev=$DEVICE SOF 

# Uses Joerg Schilling's "cdrecord" package (see its docs). 

# http://www.fokus.gmd.de/nthp/employees/schilling/cdrecord.html 

# Newer Linux distros may use "wodim" rather than "cdrecord" 


echo; echo "Done copying $OF to CDR on device $CDROM." 


echo "Do you want to erase the image file (y/n)? " # Probably a huge file. 
read answer 


CO CO CO CO CO CO CO CO CO CO PO 
Wey Keer e] (ex Gal ISS GD) INS) |S (m ike) 


a e 
i €x 


case "Sanswer" in 

[yY]) rm -f SOF 

echo "SO erased,” 
ii 


(exelayo). VSOT Mor Xexctevexerola WRA 


Pb bP PP Ss 
(Ko) Keo) cp ony (Gal HSS (OS) J55) 
+ 


50 # Exercise: 
51 # Change the above "case" statement to also accept "yes" and "Yes" as input. 


53 exit 0 


Example A-6. Collatz series 


!/bin/bash 
e@@llikaie” ea 


The notorious "hailstone" or Collatz series. 


Get the integer "seed" from the command-line. 


ar ab: Gee weols lox; S) ghayel ael il. 
NUMBER «-- result 
Loop back to step 3 (for specified number of iterations). 


1) 

2) 

33) ERE 

4) If NUMBER is even, divide by 2, or 
5) 

6) 

7) 


The theory is that every sequence, 
+ no matter how large the initial value, 
+ eventually settles down to repeating "4,2,1..." cycles, 
+ even after fluctuating through a wide range of values. 


This is an instance of an "iterate," 
* an operation that feeds its output back into its input. 


Mop pop pp PPP PY 
O (o 00 -1 O Oi i$ (). M. IB. O xo 0 -1 O O1 i8 CQ) I 


2l Sometimes the result is a "chaotic" series. 
22 

23 

24 MAX ITERATIONS-200 


N 
[91] 


# For large seed numbers (532000), try increasing MAX ITERATIONS. 


N N 
yA 


h=${1:-$$} # Seed. 
Use SPID as seed, 
#+ if not specified as command-line arg. 


N 
[o0] 
+ 


echo 
echo "C($h) --- SMAX ITERATIONS Iterations" 
echo 


for ((i=1; i<=MAX_ITERATIONS; i++)) 
do 


# echo -n "Sh y 


CO CO CO CO CO CO CO CO CO CO Dd 
«Oc Co) OE WNP oo 


# ARAN 
40 # tab 
41 # printf does it better 


42 COLWIDTH=%7d 

43 printf SCOLWIDTH Sh 

44 

45 let "remainder = h % 2" 

46 we | USsaeunsdbgpkeus" -860 © | # Even? 

47 then 

48 let "h /= 2" # Divide by 2. 

49 else 

50 Jkexe Mig Tavs} wee IE # Multiply by 3 and add 1. 
51 1E al 

5 

53 

54 COLUMNS=10 # Output 10 values per line. 


55 let "line break = i $ COLUMNS" 


56 ase [ VSiliding break ag © | 
57 then 

58 echo 

59 EL 


61 done 
63 echo 
65 # For more information on this strange mathematical function, 


66 #+ see Computers, Pattern, Chaos, and Beauty , by Pickover, p. 185 ff., 
67 #+ as listed in the bibliography. 


69 ext © 


Example A-7. days-between: Days between two dates 


!/bin/bash 
days-between.sh: Number of days between two dates. 
Usage: ./days-between.sh [M]M/[D]D/YYYY [M]M/[D]D/YYYY 


Note: Script modified to account for changes in Bash, v. 2.05b +, 
ar that closed the loophole permitting large negative 
4 integer return values. 


ARGS=2 # Two command-line parameters expected. 
E PARAM ERR=85 # Param error. 


REFYR=1600 # Reference year. 

CENTURY=100 

DIY=365 

ADJ_DIY=367 # Adjusted for leap year + fraction. 
MIY=12 

DIM=31 

,EAPCYCLE-4 


4 


Mop pop p ppppÀpÀG: 
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MAXRETVAL-255 # Largest permissible 


ZA #+ positive return value from a function. 

22 

23 diff- # Declare global variable for date difference. 
24 value- # Declare global variable for absolute value. 
25 day- # Declare globals for day, month, year. 

26 month- 

27 year- 

28 


Perce meiro O) # Command-line parameters wrong. 


echo "Usage: `basename $0` [M]M/[D]D/YYYY [M]M/[D]D/YYYY" 
exem) (Gate must be arter 173/1600) V 
exit $E_PARAM ERR 


Parse_Date () # Parse date from command-line params. 


{ 


CO CO CO CO CO CO CO CO CO CO Dd 
Wo) Xoer X] ony Gai GSS (699 [NO |=) xe ke) 


40 month=$ {1%%/**} 

AVL dm=$ {1%/**} # Day and month. 

42 day-$ {dm#*/} 

43 let "year = ^basename $1'" # Not a filename, but works just the same. 


D e 
(Om ge 


46 


47 check date () # Checks for invalid date(s) passed. 
48 ( 

AS I Teesi =e TEDT l i T Tenorea” egt m oM pn TUI 

50 L “svyear™ Sit “SREEYR" S SEA amv 5ons 

5l # Exit script on bad value(s). 

52 # Uses or-list / and-list. 

53) # 

54 # Exercise: Implement more rigorous date checking. 

55 } 

56 

57 

58 strip_leading_zero () Better to strip possible leading zero (s) 
59 ( * from day and/or month 

60 return ${1#0} + since otherwise Bash will interpret them 
DITE tee gere valves. (POSIX seb 2.922%. 4). 
62 

63 

64 day_index () Gauss! Formula: 

GENE Days from March 1, 1600 to date passed as param. 
66 QUA QUAIN XOT 

67 day=$1 


68 month=$2 
69 year-$3 


70 

WAL let "month = $month - 2" 

72; i£ | ""Smonth" —Te- 0 | 

73 then 

74 let "month += 12" 

75 jc yc a = nik 

WE iE aL 

Va 

78 let "year -= SREFYR" 

7€ let "indexyr = $year / SCENTURY" 

80 

81 

82 let "Days = $DIY*$year + Syear/SLEAPCYCLE - Sindexyr N 

83 + Sindexyr/SLEAPCYCLE + SADJ DIY*S$month/S$MIY + Sday - SDIM" 
84 # For an in-depth explanation of this algorithm, see 

85 #+ http://weblogs.asp.net/pgreborio/archive/2005/01/06/347968.aspx 


86 

87 

88 echo $Days 

89 

ORE 

31 

92 

93 calculate difference () 4 Difference between two day indices. 
94 ( 

95 leg ghi = Sil = SQ # Global variable. 

9G j 

97 

98 

99 abs () # Absolute value 

MOO) 3 # Uses global "value" variable. 
101 phe) if STR CST poe E # If negative 

1912) then #+ then 

103 lec Yvette = Q = Si #+ change sign, 

104 else #+ else 

105 let "value = $1" #+ leave it alone. 

106 fa 

dy } 

108 

LOS 

LILO 

iii ae [D Sir sixes WSvNSOS"U ] # Require two command-line params. 


Error 


Parse Date $1 
check date $day $month $year 


strip leading zero $day 
day=$? 

strip_leading_zero Smonth 
month=$? 


let "datel "day index $day Smont 


Parse Date $2 
check date $day $month $year 


strip leading zero $day 
day=$? 

strip_leading_zero Smonth 
month-$? 


calculare chitieiasncS Selsel SelsueZ 


alos SEELE 
diff-$value 
echo Sdiff 


exi 


Exercise: 


date2-$ (day index $day $month $year) 


# See if valid date. 


# Remove any leading zeroes 
#+ on day and/or month. 


h $year'/" 


# Command substitution. 


# Make sure it's positive. 


If given only one command-lin 
use today's date as the second. 


Compare this script with 
+ the implementation of Gauss' Fo 
http://buschencrew.hypermart 


Example A-8. Making a dictionary 


1 #!/bin/bash 

2 makedict.sh [make dictionary] 

3 

4 Modification of /usr/sbin/mkdict 
5 Qwigimell SewwSse Copsueaugiau 1:999. 
6 

7 This modified script included i 
8 #+ consistent with the "LICENSE" 

jer idee aie Oricalinel Semis 2S Gl ]9 
10 

ial This script processes text fil 
12 #+ of words found in the files. 

13 This may be useful for compilin 
14 #+ and for other lexicographic pur 
15 


document of the 


parameter, have the script 


rmula in a C program at 
.net/software/datedif 


(/usr/sbin/cracklib-forman) 
by Alec Muffett. 


SICJE3LOTE c 


n this document in a manner 
"Crack" package 
gue OE; 


S to produce a sorted list 


g dictionaries 
poses. 


17 E BADARGS-65 
18 
US ase [p P» sie "Si" ] # Need at least on 
20 then #+ valid file argument. 
Zl: echo "Usage: $0 files-to-process" 
22 exit $E BADARGS 
AS) arab 
24 
25 
26 4 SORT="sort" No longer necessary to define options 
2.7) 4r eO Siow. (Clawewsweyevol sccm eakepueeudl siciealjoye s 
28 
29 cat $% | Contents of specified files to stdout. 
30 ieag INS vov || Convert to lowercase. 
Sil iret mee MEOH tii New: change spaces to newlines. 
32 t tr -cd '\012[a-z][0-9]' | Get rid of everything non-alphanumeric 
$3 ae (Gua. (xedkepimwell SCCI) o 
34 tr -c '\012a-z' AO Rather than deleting non-alpha chars, 
39 + change them to newlines. 
36 Sorc | SSORT options unnecessary now. 
S7 winte || Remove duplicates. 
38 Geej cuv Ase) || Delete lines beginning with a hashmark. 
39 grea -v USS" Delete blank lines. 
40 
41 exit 0 
Example A-9. Soundex conversion 
!/bin/bash 
soundex.sh: Calculate "soundex" code for names 


Mop pop opp PPP PY 
O (o 0» -1 O Oi i$ (). M P O xo 0 -1 O O1 4 Q Io P 


(NX [R9 ISS) [m5 mo» TNS [e 
e| (ex Gal dex (o IS) S 


N 
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WWW N 
NEO 


33 


Soundex script 

by 
Mendel Cooper 
thegrendel.abs@gmail.com 
reldate: 23 January, 


A slightly different version 
+ Ed Schaefer's July, 2002 
vaio REVI (Xm: 


ap am 


2002 


"Shell Corner" 


Placed in the Public Domain. 


of this script appeared in 
column 


+ http://www.unixreview.com/documents/unil026336632258/ 


ARGCOUNT-1 
E WRONGARGS-90 


ame [[ 

then 
echo "Usage: '"basename $0" 
exit $E WRONGARGS 

ita 


$4 -ne "SARGCOUNT" ] 


0 


assign value 


{ 


vall=bfpv 


# Need name as argument. 


name" 


# Assigns numerical value 
#+ to letters of name. 


i^ "ew wl = db 


34 
35 
36 
3r 
38 
EI 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sil 
52 
55 
54 
55 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
pit 
p 
13 
74 
WS 
76 
p 
78 
US 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
21 
92 
23) 
94 
95 
96 
2 
98 
99 


val2=cgjkqsxz dou SOTA Kor ASAA | =u 
val3=dt to ete: 

val4=1 

val5=mn 

val6=r 


Exceptionally clever use of 'tr' follows. 
Try to figure out what is going on here. 


value=$( echo "$1" \ 

tr -d wh \ 

im Sywalli d | te Swel2 2 | exe Swails 3 \ 
ie Swell 4L | tee SyalS 5 | iue svali 6 \ 
tr -s 123456 \ 

iE Cl aSiOUny )) 


Assign letter values. 

Remove duplicate numbers, except when separated by vowels. 
Ignore vowels, except as separators, so delete them last. 
Ignore 'w' and 'h', even as separators, so delete them first. 


The above command substitution lays more pipe than a plumber <g>. 


input name-"$1" 


echo 
echo "Name = $input name" 
Change all characters of name input to lowercase. 
name=$( echo S$input name | tr A-Z a-z ) 
Just in case argument to script is mixed case. 
Prefix of soundex code: first letter of name. 
char. pos-0 # Initialize character position. 
prefix0-$(name:$char pos:1) 


prefix= echo Sprefix0 | tr a-z A-Z 
# Uppercase lst letter of soundex. 


ike, Weloee isos: crm dis # Bump character position to 2nd letter of name. 
namel-$(name:$char. pos) 


WP ceupepaRSeGRSPARTRGRSPH PSU EYPA RS HR SPA R GR SPAeSe y dücx eqpabdLowa Paie spseaedbapseqeaparseaeaearsedeararseaeaearsedeararse ae sears 
# Now, we run both the input name and the name shifted one char 

#+ to the right through the value-assigning function. 

# If we get the same value out, that means that the first two characters 
#+ of the name have the same value assigned, and that one should cancel. 

# However, we also need to test whether the first letter of the name 

#+ is a vowel or 'w' or 'h', because otherwise this would bollix things up. 


Chari= echo Soresfiz | stus A-2 mm # First letter of name, lowercased. 


assign_value $name 
sl=Svalue 
assign value Snamel 
s2=Svalue 
assign value Scharl 


s3=Svalue 
s3=9Ss3 # If first letter of name is a vowel 
ump ene Ui ene Vin 
#+ then its "value" will be null (unset). 
#+ Therefore, set it to 9, an otherwise 
#+ unused value, which can be tested for. 


iat [pp “Sel? sme VUSg2w [| Wess" ee S ]] 
then 
suffix-$s2 
else 
suffix-$(s2:$char pos) 
To. 


# ++++++++++4+4+4+4+4+4+4+4+4+4+4++ end Exception Patch ++++++++++4+4+4+44+444444444444+ 


padding=000 # Use at most 3 zeroes to pad. 
soun=SprefixSsuffix$padding # Pad with zeroes. 
MAXLEN=4 # Truncate to maximum of 4 chars. 


soundex=S {soun:0:SMAXLEN} 


echo "Soundex = $soundex" 


echo 


The soundex code is a method of indexing and classifying names 
+ by grouping together the ones that sound alike. 
The soundex code for a given name is the first letter of the name, 
+ followed by a calculated thr number cod 
Similar sounding names should have almost the same soundex codes. 


Examples: 

Smith and Smythe both have a "S-530" soundex. 
Harrison = H-625 

Hargison IB (5272 

Harriman - H-655 


s 
1 


his works out fairly well in practice, but there are numerous anomalies. 


The U.S. Census and certain other governmental agencies use soundex, 
as do genealogical researchers. 


Por more information, 
+ see the "National Archives and Records Administration home page", 
* http://www.nara.gov/genealogy/soundex/soundex.html 


# Exercise: 


y Salujollaliey ijs Widpeisjoit ito Ieri! exei (xr ills SI(esetels a 


exatic O 


Example A-10. Game of Life 


il 


#!/bin/bash 


NPRPRPPRP PPP PY 
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CO CO CO CO CO CO CO CO CO CO Dd 
We) (99). I x) (On des ta) [SS TS) Sy wo 


Or FPP WA be PP aumÉ us us 
f£». or (69 Jp oy (Ga DS (3. [R3 ie SS 
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Version 0.2: Patched by Daniel Albers 
+ to allow non-square grids as input. 
Version 0.2.1: Added 2-second delay between generations. 


HHP HEHE HHT HEE HH HEHE HE HEHE EH HH HEHE HE EE EH EH EE TE EE TE EEE EE EEE HE EE EH 
This is the Bash script version of John Conway's "Game of Life". 
"Life" is a simple implementation of cellular automata. 


On a rectangular grid, let each "cell" b Loe Waly ie WoE 
Designate a living cell with a dot, and a dead one with a blank space. 
Begin with an arbitrarily drawn dot-and-blank grid, 
+ and let this be the starting generation, "generation 0." 
Determine each successive generation by the following rules: 
1) Each cell has 8 neighbors, the adjoining cells 
* left, right, top, bottom, and the 4 diagonals. 


1,25 
Ales 5; Ios: = is Gas Cell trier Geis alcleieene a Cin, 
678 


2) A living cell with either 2 or 3 living neighbors remains alive. 
SURVIVE=2 

3) A dead cell with 3 living neighbors comes alive (a "birth"). 
BIRTH=3 

4) All other cases result in a dead cell for the next generation. 

HPT HE EEE EE EEE HE HE EEE EE EH HE HE ERE HERE EE ERE ER HE EEE EE EE HE EE HE EE HF 


startfile=gen0 # Read the starting generation from the file "gen0" 
# Default, if no other file specified when invoking script. 
# 

ie [| =i SG | # Specify another "generation 0" file. 

then 


startfile="$1" 
i3 


FERE FE HE TE FE FE HE E EEE HE EE EEE EE HE HE EERE EE E E E HE EEE 
Abort script if "startfile" not specified 
* and 

+ default file "gen0" not present. 


E NOSTARTFILE-86 


ads [qp P e VSisiceweirtiile” |] 

then 
Scho "SibeEfile YUUEgcesEE eU miselne 
exit SE NOSTARTFILE 


fed 
HH HE EEE RE EE ERE ERE EH EE EE EEE RE EE HEE EE EEE 
ALIVE1=. 
DEAD1=_ 
# Represent living and dead cells in the start-up fil 
# # 
# This script uses a 10 x 10 grid (may be increased, 
#+ but a large grid will slow execution). 
ROWS=10 
COLS=10 
# Change above two variables to match grid size, as desired. 
# # 


GENERATIONS=10 # How many generations to cycle through. 


m 
WO 


# Adjust this 
#+ if you have 


upwards 
time on your hands. 


NONE_ALIVE=85 # Exit status on premature bailout, 
jar abie solo) Celile lere allies, 

DELAY=2 # Pause between generations, etc. 

TRUE=0 

FALSE=1 

ALIVE=0 

DEAD=1 

avar= # Global; holds current generation. 

generation=0 # Initialize generation count. 

# 

let "cells = SROWS * SCOLS" # How many cells. 

# Arrays containing "cells." 


Cleellaias =a aliniavie ae 


declar a current 
display () 

{ 

alive=0 

declare -a arr 


euex-—( "exeo USIW" ) 


element count-$(£sarr[*]) 


hoe alate 
local rowcheck 


for ((i=0; 


do 


# Insert newline at 


i<Selement_count; 


ja(ol ie 


# How many cells alive at any given time. 


# Initially zero. 


# Convert passed arg to array. 


i++) ) 


ach row. 


lee Vroycheck = Sa. 
ii T 
then 
echo 
xoleo) ior W X 
a 


cell=${arr[i]} 


aie {l 
then 

let 
i 


Nice — S] 
"alive += 1" 
echo -n "$cell" 

# Print out array, 


done 


return 


IsValid () 
( 
if [ -z 


usq ui To 7 


"Srowcheck" -eq 0 


Sco 
changing underscores to spaces. 


WS ] 


COLS" 


] 


# Newline. 
# Indent. 


= Ug y fe 


# Test whether cell coordinate valid. 


# Mandatory arguments missing? 


134 
LSS 
136 
LS 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
pil 
152 
153 
154 
155 
156 
157 
158 
159 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
JUL 
172 
LAS) 
174 
LYS 
176 
Wa 
178 
LPS 
180 
181 
182 
183 
184 
185 
186 
ISN 
188 
189 
190 
19 
192 
193 
194 
195 
196 
119g 
198 
199 


then 
return SFALSE 
ita 
local row 
local lower limit-0 # Disallow negative coordinate. 
local upper_limit 
loca ed 
Jhexeeul 3eabgnu 
let "upper limit = SROWS * SCOLS - 1" # Total number of cells. 
sue [ Wi" ike "Siloses loni" =0 "SL" e "Suppe Jaw | 
then 
return SFALSE # Out of array bounds. 
ica 
row-$2 
lec "lei = Sau  SXCOLS" ur bee disini. 
lies; Wieigine = Siler se SCOS — dV # Right limit. 
TE [ "ea" -1t ONS TEIE —Oo moque -GT VSee ] 
then 
return SFALSE # Beyond row boundary. 
ia 
return STRUE # Valid coordinate. 
} 
IsAlive () # Test whether cell is alive. 
# Takes array, cell number, and 
{ #+ state of cell as arguments. 
GecCounne Visit S2 # Get alive cell count in neighborhood. 
local nhbd=$? 
if [ "$nhbd" -eq "SBIRTH" ] # Alive in any case. 
then 
return SALIVE 
dE ak 
is [ "S3" = "UV =e Uude" ee USSUEVAVEC ] 
then # Alive only if previously alive. 
return SALIVE 
ie 


return SDEAD 


GetCount () 


cell number-$2 
array 

top 

Sener 

bottom 


# Dead, by default. 


# Count live cells in passed cell's neighborhood. 
# Two arguments needed: 

variable holding array 

cell number 


NONNNNNN PS 


N 
Ko) 


next_gen () 


{ 


Toeall 36. 18/919) 
local t cen 
Loew). t 90E 
local count=0 
local ROW_NHBD=3 
antay- (echon yoM) 
Je "Eg = Scell sawaorsc = SCOLS = i 
let "center = $cell number - 1" 
ler YWoociconm = Seeilil_imunilsyese ++ SCOLS = IY 
le “We = cell minbar / SCOLS” 
for ((i20; i<SROW_NHBD; i-*-)) 
do 
let "t top = Stop + $i" 
lee We exem = Secemicere a San 
let iE. Jo. = Slooicirem < Si! 
let "row = $r" 


do 


IHE 


th 


Fa 


PE 


IsValid $t cen Srow 


Set up cell neighborhood. 


Traverse from left to right. 


Count center row. 
Valid cell position? 


Is it alive? 
If yes, then 
Increment count. 


(P OUnteisc IMS OWI 


Redundancy here. 
Can be optimized? 


Count bottom row. 


of tested cell itself 


if [ $? -eq "STRUE" ] 
then 
sie (| Stama Se Cem = Sb yim 3p sm 
then i 
Vet "count +=" T" # 
$E 3L 
sen 
etant ow c Sues c LU # 
IsValid $t top $row 
ii | $9 -ee USE | 
then 
if [ $(array[S$t colk = "SALIVE1" ] f£ 
then i 
dex: 'exonbliaue. a db 
Lx 
sen 
ete "reg = Sie sp LU Ls 
IsValid $t bot Srow 
xi | SO es VST? Í 
then 
se [| Seray lee ooe] = USAGE! T 
then 
let VexoxbueuE sr" iL 
Hee 
$E 3b 
ne 
[ ${array[$cell_number]} = "SALIVE1" ] 
en 
let "count -= 1" # Make sure valu 
#+ is not counted. 
turn $count 


# Update generation array. 


CO) CO CO CO CO CO CO CO 


WW 
Nh + 
(oy We) 


local array 
local i=0 
array=( “echo "$1" ) # Convert passed arg to array. 
wase [ WS i WSeeiikew T 
do 
isAlive YIU Si Sese [Sil] } 3 xs cell oliye? 
zie [ S9 ee "Sangue" T 
then sr JOE. gba "loco 
array[$i]-. #+ represent the cell as a period. 
else 
ankay kon NT # Otherwise underscor 
E #+ (will later be converted to space). 
Vou mr. — Nn 
done 
# let "generation += 1" # Increment generation count. 
# Why was the above line commented out? 
# Set variable to pass as parameter to "display" function. 
avar= echo S{array[@]}° # Convert array back to string variable. 
display "Savar" # Display it. 
echo; echo 
echo "Generation $generation - Salive alive" 
wie || Wewulsbyew Sexe, © T 
then 
echo 
echo "Premature exit: no more cells alive!" 
exit SNONE_ALIVE j^ No porne im (eoe aliowyaL save 
fak #+ if no live cells. 
} 
main () 
Load initial array with contents of startup file. 
initial=( "cat "S$startfile" | sed -e "/#/d' | tr -d 'Nn' [N 
Delete lines containing '#' comment character. 
Geol Ue OWL XS 7g cum Vui M. we ) 
Remove linefeeds and insert space between elements. 
clear # Clear screen. 
echo # Title 
Solem CSVC ON 
cho Ww "n 
setterm -reverse off 
echo " SGENERATIONS generations" 
echo " (yi lU 
exciso) Wisse im Coe Sube! Jueuaxery V 
Severe -revere om 
cho Ww Ww 
setterm -reverse off 
Sleep SDELAY 4 Display "splash screen" for 2 seconds. 
Ub ==> Display i&:3e8iE Generation, ===- 
Gen0-'echo $(initial[GQ])' 


S332 
33S) 
334 
SIS 


display "S$GenO" 
echo; echo 

echo "Generation $generation 
Sleep SDELAY 


# Display only. 


Salive alive" 


336 
S3 
398) 
38g 
340 
341 
342 
343 


# 


"generation += 1" 


Cur= echo $[initial[Q8])^ 


Display second generation. 


# Bump generation count. 


Saa met gen "US # Update & display. 

345 sleep $DELAY 

346 # 

347 

348 let "generation += 1" # Increment generation count. 

349 

350 $6 ————-— Main loop for displaying subsequent generations ------ 
351 while [ "$generation" -le "SGENERATIONS" ] 

S52 Clo) 

953 Cur="Savar" 

354 ies exero. VSC 

355 let "generation += 1" 

356 sleep SDELAY 

357 done 

35g s 

259 

360 echo 

361 

362 exit 0 # CEOF:EOF 

363 

364 

365 

366 The grid in this script has a "boundary problem." 

367 The the top, bottom, and sides border on a void of dead cells. 
368 Exercise: Change the script to have the grid wrap around, 

369 t so that the left and right sides will "touch," 

370 T as will the top and bottom. 

evel 

S Exercis Create a new "gen0" file to seed this script. 

33 Use c. 112 s« IG Gieh imsicsecl Qut Hae qcxe3ieiuevedE dH) «x 10 ome, 
374 Make the necessary changes to the script, 

SIS. uer SO it will run with the altered fil 

SHE 

S71 Exercise: Modify this script so that it can determine the grid size 
378 #+ from the "gen0" file, and set any variables necessary 
SHS) sr iQ iEloXe script "ior ieibuol. 

380 This would make unnecessary any changes to variables 
Geb ger in the script for an altered grid size. 

382 

383 Exercise: Optimize this script. 

384 It has some redundant code. 


Example A-11. Data file for Game of Life 


geno 


This is an example "generation 0" 


Staco—we itil tore Wnte, sia". 


The "gen0" 
+ and an underscore (_) 


(ox, Cal Wes (E53. fu». (3 


file as s dh) x 10) qge3uel wusabe a Teese (65) 
for dead ones. 


for live cells, 
We cannot simply use spaces 


7 #+ for dead cells in this file because of a peculiarity in Bash arrays. 
8 # [Exercise for the reader: explain this.] 
9 # 

10 # Lines beginning with a '#' are comments, and the script ignores them. 
dies A oe trien cue 

T2 ee 

13 — 

14 a 

15 

16 uc pe 

L7 

le o 

OS RN 

Zope E. 

+++ 


[GHT for 


The following script is by Mark Moraes of the University of Toronto. See the file Moraes-COPYR] 


permissions and restrictions. This file is included in the combined HTML/source tarball of the ABS Guide. 


Example A-12. behead: Removing mail and news message headers 


! /bin/sh 

Strips off the header from a mail/News message i.e. till the first 
empty line. 

Author: Mark Moraes, University of Toronto 


==> These comments added by author of this document. 


if [ S# -eq 0 ]; then 


NOB pop p PPP PP 
O (o 0» -1 O Ui i (). M HIS. O (o 0 -1 O O1 4 Q I E 


sec -e Ud/^8/6 e "JF ] 58/8" 
# —-> Delete empty lines and all lines until 
# --» first one beginning with white space. 
else 
# ==> If command-line args present, then work on files named. 
tor i do 
sec =e "L./^S5/gd" =e "J^l| ]*9/e" Sat 
ub =——S Ditto, BS ciens 
done 
ical 
21 exit 
BD 
23 # ==> Exercise: Add error checking and other options. 
24 4 ==> 
25 4 ==> Note that the small sed script repeats, except for the arg passed. 
26 4 ==> Does it make sense to embed it in a function? Why or why not? 
27 
28 
29 ps 
30 > Copyrcigue Umiversicy goi Noromeo i999, 1999. 
31 * Written by Mark Moraes 
m € 
33 * Permission is granted to anyone to use this software for any purpose on 
34 * any computer system, and to alter it and redistribute it freely, subject 
35 = to the following restrictions: 
9o = 
37 * 1. The author and the University of Toronto are not responsible 
SO E for the consequences of use of this software, no matter how awful, 
99 8 even if they arise from flaws in it. 
* 


od 
ce 


==> If no command-line args present, then works on file redirected to stdin. 


41 * 2. The origin of this software must not be misrepresented, either by 
22 explicit claim or by omission. Since few users ever read sources, 
ALS Xs credits must appear in the documentation. 

dd o 

45 * 3. Altered versions must be plainly marked as such, and must not be 
46 * misrepresented as being the original software. Since few users 

47 * ever read sources, credits must appear in the documentation. 

a a 

49 * 4, This notice may not be removed or altered. 

50 =/ 


+ 


Antek Sawicki contributed the following script, which makes very clever use of the parameter substitution 
operators discussed in Section 10.2. 


Example A-13. password: Generating random 8-character passwords 


!/bin/bash 
May need to be invoked with #!/bin/bash2 on older machines. 


Random password generator for Bash 2.x + 
+ by Antek Sawicki <tenox@tenox.tc>, 
+ who generously gave usage permission to the ABS Guide author. 


==> Comments added by document author ==> 


MATRIX-"0123456789ABCDEFGHIJKLMNOPORSTUVWXYZabcdefghijklmnopqrstuvwxyz" 


# ==> Password will consist of alphanumeric characters. 
LENGTH="8" 
# ==> May change 'LENGTH' for longer password. 


NPRPRPRPP PPP PY 
SCOMIDTKRWNHFPOWCMBIDGURWNE 


while [ "S{n:=1}" -le "SLENGTH" ] 
# ==> Recall that := is "default substitution" operator. 
a ==> Go, cic Um! luae mor been imitializec, Seic ate to I. 
do 
2 PASS="S$PASS${MATRIX:$ ( (SRANDOM%S { #MATRIX})):1}" 
22 ==> Very clever, but tricky. 
23 
24 ==> Starting from the innermost nesting... 
25 ==> S${#MATRIX} returns length of array MATRIX. 
26 
27) ==> SRANDOM%SS{#MATRIX} returns random number between 1 
28 ==> and [length of MATRIX] - 1. 
29 
30 ==> S$(MATRIX:$((SRANDOM$S(4MATRIX))):1) 
Si ==> returns expansion of MATRIX at random position, by length 1. 
32 ==> See {var:pos:len} parameter substitution in Chapter 9. 
33) ==> and the associated examples. 
34 
35) ==> PASS=... simply pastes this result onto previous PASS (concatenation). 
36 
s ==> To visualize this more clearly, uncomment the following line 
38 echo "SPASS" 
S9 ==> to see PASS being built up, 
40 ==> one character at a time, each iteration of the loop. 
41 
42 let n+=1 
43 ==> Increment 'n' for next pass. 


44 
45 
46 
47 
48 


done 
exelmo VSPAGSY # ==> Or, redirect to a file, as desired. 


vae (0) 


+ 


James R. Van Zandt contributed this script which uses named pipes and, in his words, "really exercises 
quoting and escaping." 


Example A-14. fifo: Making daily backups, using named pipes 


NPRPRPPP PPP PY 
O (o 00 -1 O Oi i (). NN HP. O xo 0 -1 O O1 4 CQ Io ES 


NNNNN DN ND 
=] (x (On HSS Gs) IS) [55 


N 
œ 


CO CO CO CO NO 
(Goi [n5 [SY «x We) 


34 


se HE 


!/bin/bash 
==> Script by James R. Van Zandt, and used here with his permission. 


==> Comments added by author of this document. 


HERE= uname -n` # ==> hostname 

THERE=bilbo 

echo "starting remote backup to $THERE at "date +%r°" 

i? SSS eera uestre? aA eis eub ali, JL —JoXoxbnc oane bec. WOSIOSssa weil. 


# make sure /pipe really is a pipe and not a plain file 
xo eE iE 


mkfifo /pipe # ==> Create a "named pipe", named "/pipe" 
# ==> 'su xyz' runs commands as user "xyz". 
# ==> 'ssh' invokes secure shell (remote login client). 


su xyz e "ssh STHERE \"cat > /home/xyz/backup/S(HERE)-daily.tar.gzN" 


tar -czf - bin boot dev etc home info lib man root sbin share usr var 
==> Uses named pipe, /pipe, to communicate between processes: 
==> 'tar/gzip' writes to /pipe and 'ssh' reads from /pipe. 


==> Th nd result is this backs up the main directories, from / on 


==> What are the advantages of a "named pipe" in this situation, 
==>+ as opposed to an "anonymous pipe", with |? 
==> Will an anonymous pipe even work here? 


==> Is it necessary to delete the pipe befor SIE LING] (Ele GXeseiugiE 2 
==> How could that be done? 


exci O 


< /pipe"& 


> /pipe 


down. 


+ 


Stéphane Chazelas used the following script to demonstrate generating prime numbers without arrays. 


Example A-15. Generating prime numbers using the modulo operator 


1 #!/bin/bash 
2 # primes.sh: Generate prime numbers, without using arrays. 
3 # Script contributed by Stephane Chazelas. 


This does *not* use the classic "Sieve of Eratosthenes" algorithm, 
+ but instead the more intuitive method of testing each candidate number 
3r rO ratos ((CliwalS@~s)) y Veling ise Wea! mochila (Oqexenetceuc(oye c 


LIMIT-1000 i? Dismiss, 2 sa OOO. 
Primes () 
{ 

((( 3m = Sal ab ») # Bump to next integer. 
SAILE # Next parameter in list. 


$?o eeno " wm i81 V 


(so m H H H H H H H 1 
@) Wer (tec) c] fex, Gl dex. GD NS [A SS We) (e9 s ix; Gal des 


we (om S= mad jj 
then echo $* 
return 
ike THERE 
22 
Ao eres GO # "i" set to "@", previous values of $n. 
24 # cho "-n-$n i-$i-" 
25) (Cs 3. ak ug Jb) xS break # Optimization. 
26 ((n$i)) && continue # Sift out non-primes using modulo operator. 
27 Primes $n $Q # Recursion inside loop. 
28 return 
2 done 
30 
31 Primes $n $0 $n # Recursion outside loop. 
32 #  Successively accumulate 
ES #+ positional parameters. 
34 4 "S@" is the accumulating list of primes. 
S. } 
36 
37 Primes 1 
38 
39 exc Gm? 
40 
41 Pipe output of the script to 'fmt' for prettier printing. 
42 
43 Uncomment lines 16 and 24 to help figure out what is going on. 
44 
45 Compare the speed of this algorithm for generating primes 
46 #+ with the Sieve of Eratosthenes (ex68.sh). 
47 
48 
49 Exercise: Rewrite this script without recursion. 


+ 


Rick Boivie's revision of Jordi Sanfeliu's tree script. 


Example A-16. tree: Displaying a directory tree 


1 #!/bin/bash 

2 tree.sh 

3 

4 Written by Rick Boivie. 

5 Used with permission. 

6 This is a revised and simplified version of a script 

7 #+ by Jordi Sanfeliu (the original author), and patched by Ian Kjos. 
8 This script replaces th arlier version used in 

9 #+ previous releases of the Advanced Bash Scripting Guide. 

10 Copyright (c) 2002, by Jordi Sanfeliu, Rick Boivie, and Ian Kjos. 


# ==> Comments added by the author of this document. 


hs) [E [Eq qt [1S4 tit 
@ “or (oor «| (xy Gal dex wo IS [A 


search () { 
itgue Clie iim echo ^ 
# ==> "echo * lists all the files in current working directory, 
#+ ==> without line breaks. 
# ==> Similar effect to for dir in * 
# ==> but "dir in “echo **" will not handle filenames with blanks. 
2L oko) 
22 ig (ob Seba qp 4 Aelia ==> IE ads aks @ oBmureeuouw (9) ase 
23 = ==> Temp variable, keeping track of 
24 directory level. 
2:5) Wine, |p Sx f Sil T Keep track of inner nested loop. 
26 do 
2:3) Schot cum Wn ow --» Display vertical connector symbol, 
28 ==> with 2 spaces & no line feed 
29 in order to indent. 
30 gig SSqoie a ap db ==> Increment zz. 
Sal done 
32 
38 a dp Sik Uh ] 6 egan vs SSS dte o Setor abet xev Syyamloyollate: Aaaa 
34 echo "4---$dir" ^ls -1 $dir | sed 's/^.*'S$dir' //"' 
35 # ==> Display horiz. connector and list directory name, but... 
36 # ==> delete date/time part of long listing. 
37 else 
38 echo Vp Sese # ==> Display horizontal connector symbol... 
39 # ==> and print directory name. 
40 numdirs-'expr $numdirs + 1' # ==> Increment directory count. 
41 tE (exgi WiseliseW p. edem # ==> If can move to subdirectory... 
42 search ~expr $1 + 1° # with recursion ;-) 
43 # ==> Function calls itself. 
44 cd 
45 iB aL 
46 3E 3L 
47 iral 
48 done 
He) } 
50 
Sil ase [ Sy de 0 ] p oiim 
52 eel Sil # Move to indicated directory. 
59 #else # stay in current directory 
54 fi 
55 
56 echo "Initial directory = “pwd " 
57 numdirs=0 
58 
59 search 0 
60 echo "Total directories = $numdirs" 
61 
62 exit 0 


Patsie's version of a directory tree script. 


Example A-17. tree2: Alternate directory tree script 


#!/bin/bash 
# tree2.sh 


1 

2 

3 

4 # Lightly modified/reformatted by ABS Guide author. 

5 # Included in ABS Guide with permission of script author (thanks!). 
6 


NPRPRPRPRPP PP PY 
O (o 0 -1 O OU): ( M P O «o 0 -1 


(69) (EO) (SS) [Sr ASS) [RO fs9x TRS? ASO} [eS de» 
[= «53 Wo) (e| ss) (x (nl SS Ge PS) TY 


w 
N 


C0 CO CO CO CO CO CO 
OMIA Bf Ww 


40 


oe 
bh 


Pm Pb SP PP am aus 
«oO 00-10) 015 CQ IN 


or O1 
e O 


(nk Gah (Om 
B wW ND 
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oo om ul 
Oo O —1 Ov 


(opp iem (63. (6n 46x) 93). ©) 
oy Gi dex (eX I) iS 


SSI yy fen) 1o» 
(S) Wey foo) 3] 


~~ 
IS [= 


S 
T 


# 


Recursive file/dirsize checking script, by Patsie 


and processes this list to a human readable tr shap 
The 'du -akx' is only as good as the permissions the owner has. 


This script builds a list of files/directories and their size (du 


akx) 


So preferably run as root* to get the best results, or use only on 
directories for which you have read permissions. Anything you can't 


reac GG} AOC ahi icles) SE 


* ABS Guide author advises caution when running scripts as root! 


HetHHEHHHE THIS IS CONFIGURABLE dB S T E 


Top 5 biggest (sub)directories. 

ax 5 subdirectories/recursions deep. 
Blank line already returned. 
Directory not specified. 


Se oc ch ok 


JTEddEESSUS DON'T CHANGE ANYTHING BELOW THIS LINE ########4# 


PID-$$ # Our own process ID. 
ELF= basename $0" # Our own program name. 
MP-"/tmp/S(SELF).S(PID).tmp" # Temporary 'du' result. 


Convert number to dotted thousand. 


function dot ( echo " gw | 


# 


eed ce se E "gJW [9-9] NX» CII0-9 NASA) 7 Ni, NAP pica” 


ieu =e 2s ] 


Usage: tree <recursion> «indent prefix» «min size» «directory» 


function tree { 


recurs="$1" 
prefix="$2" 
minsize="$3" 
dirname="$4" 


How deep nested are we? 

What do we display before file/dirname? 
What is the minumum file/dirsize? 

Which directory are we checking? 


Se SE OSE OSE 


Get (STOP) biggest subdirs/subfiles from TMP file. 
Iber eeneego “|| || esses ||] Siichisincina i / ^4 lu Uie | 

awk Se (SIS! SurawesLzs) josie e V || sort ~ne || Rhead =STOF 
[ =4 MSS! J me rerum # Empty list, then go back. 


cnt-0 
owne Sela ASKELIA || que IL” # How many entries in the list. 


## Main loop 
Scho VSS || walle regc gize nams? Cle 
( (ere rb) y f Count entry number. 


bname-'basename "$name"^ # We only need a basename of th 
[ -d "Sname" ] && bname="Sbname/" 


Hie ys 


# If it's a directory, append a slash. 


echo "‘dot $size’ $prefix +-Sbname" 
# Display the result. 
Call ourself recursively if it's a directory 
+ and we're not nested too deep ($MAXRECURS). 
The recursion goes up: $((recurs*1)) 
[The prefix gets a space if it's the last entry, 
+ or a pipe if there are mor ntries. 
The minimum file/dirsize becomes 
ra cench (ur hie parents S (este 119) ) « 
Last argument is the full directory name to check. 
abad -d "Sname" -a Srecurs -lt SMAXRECURS ]; then 
I Semne ie Sie T y 
ll rs Peces Spr ei ES sue ODE Same! 
Ge ree o (recurs l)i VSprerirs || orsiz 1:0). Sister e) 


73 itat 


74 done 

15 

76 $9 eei 0 | < echo “ SOTEN 

72) Every time we jump back add a 'blank' line. 

78 return $E BL 

1/9) We return 80 to tell we added a blank line already. 
80 } 

81 

82 ## ## 

83 ## main program ## 

84 ## ## 

85 

86 rootdir="S@" 

Sy || sel WSieececliiie gy | 

88 { echo "SSmus WSs SS LCST 29 xe fem Jane ji 
89 # We should be called with a directory name. 


90 
91 echo "Building inventory list, please wait ..." 
92 Show "please wait" message. 
SS (hui ~ak WSieooreichie” Tp» SM 5 ees iaa L1 
94 Build a temporary list of all files/dirs and their size. 
O5 ue isl =i KESTMA | eile Y {jorestinie. Sal} v^ 
96 What is our rootdirectory's size? 
97 echo "dot $size’ $rootdir" 
98 Display rootdirectory's entry. 
99) reS (9 WW 0 WERO EEN 
100 Display the tree below our rootdirectory. 
101 
LODZ ica VSTE 2// oes ausu E 
103 Clean up TMP file. 
104 


105 eave S? 


Noah Friedman permitted use of his string function script. It essentially reproduces some of the C-library 
string manipulation functions. 


Example A-18. string functions: C-style string functions 


!/bin/bash 


string.bash --- bash emulation of string(3) library routines 
Author: Noah Friedman <friedman@prep.ai.mit.edu> 

==> Used with his kind permission in this document. 
Created: ILS) 21 (9) 7/- (0 HE 

Last modified: 1993-09-29 

Public domain 


Conversion to bash v2 syntax done by Chet Ramey 


Commentary: 
Code: 


RClolelsiciailine; Piciceleie e 
Usage: strcat sl s2 


Strcat appends the value of variable s2 to variable sl. 


NPRPRPPRP PPP PY 
SCOMIDGTHRWNHHFPOWOAIDRGUARWNE 


Example: 
2 a="foo" 
22 b="bar" 
As} strcat a b 
24 echo $a 


25 d -» foobar 


26 # 

27 #:end docstring: 

28 

29 ###;;;autoload ==> Autoloading of function commented out. 
30 ituumeiEieum sECeE () 

SUL i 

32 toeau Gul val 2 velk 

33 

34 S1 val-$(!1) # indirect variable expansion 
315 s2 val-$(!2) 

36 eval USIUZXVUUSISTI valls {s2 wed] 

Sl # ==> eval $1='S${sl_val}${s2_val}' avoids problems, 

38 # ==> if one of the variables contains a single quote. 
399 jj 

40 

AL JP BCIOCRIEISIING) ISIEJCIANGISNE S 

42 Usogeswstiuca Sl S2 So 

43 

44 Line strcat, but strncat appends a maximum of n characters from the value 
45 of variable s2. It copies fewer if the value of variabl s2 is shorter 
46 than n characters. Echoes result on stdout. 

47 

48 Example: 

49 a-foo 

50 b-barbaz 

Sl Sigel cm. le S 

52 echo $a 

53 => foobar 

54 

55 #:end docstring: 

56 

57 ###;;;autoload 

De cter ion series. (0) 


59 1 

60 local si1-"$1" 

61 local g2="S2 

62 ihexeail sal soc eps 

63 ileal dL Ave SA wed 

64 

65 sl_val=${!s1} # ==> indirect variable expansion 
66 s2_val=S{!s2} 

67 

68 if [ ${#s2_val} -gt ${n} ]; then 

69 s2 val-$(s2 val:0:$n) # ==> substring extraction 

70 if aL 

31 

"e ewe Ygs W- vetal val SSA wed ur 

73 # ==> eval $1='S{sl_val}${s2_val}' avoids problems, 

74 # ==> if one of the variables contains a single quote. 

73.3 

76 

VI a OCS TN SEENE 

78 Usage: strcmp $s1 $s2 

38) 

80 Strcmp compares its arguments and returns an integer less than, equal to, 
81 or greater than zero, depending on whether string sl is lexicographically 
82 less than, equal to, or greater than string s2. 

83 #:end docstring: 

84 

85 ###; 7; autoload 

36 UMC ELOGM sercema (() 


So | 

88 [| YS! e WS2W ] Ge sew. O) 

89 

90 p Usha veu STU qx JAekssu/ioRPLl qs seein ud 


(n 
«o 


return 1 


:docstring strncmp: 
Usage: strnemp Sel Ss2 sn 


Like strcmp, but makes the comparison by examining a maximum of n 
characters (n less than or equal to zero yields equality). 
:end docstring: 


##; 7; autoload 

iriure. STANCI ()) 

{ 

ai [| oe» VSS Se. S919 Sie: ON pe Seeya 
return 0 

i3 


if [ ${3} -ge ${#1} -a $(3) -ge ${#2} ]; then 
Stren "Su Uu 
return $? 
else 
s1=${1:0:$3} 
S2=8(250283} 
strcmp SSL $s2 
return $? 
dab 


:docstring strlen: 
Usage: strlen s 


Strlen returns the number of characters in string literal s. 
:end docstring: 


Se cb che cb H 


###; ;;autoload 

unacieiOin, Ex dex (0) 

( 
eval echo "\S{#S{1}}" 
# ==> Returns the length of the value of the variable 
# ==> whose name is passed as an argument. 


:docstring strspn: 
Usage: strspn Sel $s2 


Strspn returns the length of the maximum initial segment of string sl, 
which consists entirely of characters from string s2. 
:end docstring: 


##;;;autoload 
RUNE EONMES MASON) 


{ 


# Unsetting IFS allows whitespace to be handled as normal chars. 
logal Jos 
local result-"$(1$$[!$(2)]]*)" 


echo ${#result} 
:docstring strcspn: 
Usage: strcspn $s1 $s2 
Strcspn returns the length of the maximum initial segment of string sl, 


which consists entirely of characters not from string s2. 
:end docstring: 


Se oce che ch cod 


###; 7; autoload 
function strcspn () 


{ 


# Unsetting IFS allows whitspace to be handled as normal chars. 


local IFS= 
local rsenlet=vs 1s 


le 


LS 3525] 5 


echo ${#result} 


scoeste SIETESIEIES 
Usages fug Sil s 


41 BLING; (Gt zero lancei; Siewsitic echoes Sil, 
:end docstring: 


t##; 7; autoload 
HUNG ILO SieIeSicse (0) 


{ 


# if s2 points to a string of zero length, strstr echoes sl 


| S42 -ec © ] && { eco "SI" p» roria Op } 


# strstr echoes nothing if s2 does not occur in sl 
Case WISI ta 


UON By 
wy vecury 149 
esac 


# use the pattern matching code to strip off the match and everything 


# following it 
ias S2 


# then strip off the first unmatched portion of the string 


echo "S${1##S$first}" 


ROloesE ume] SECOLE 
Usage: strtok sl s2 


BEING! logsixesmgra 


$4;;;autoload 
Punc eron SEE. (0) 


Strtok considers the string sl to consist of a sequence of zero or more 
text tokens separated by spans of one or more characters from the 
separator string s2. The first call (with a non-empty string sl 
Specified) echoes a string consisting of the first token on stdout. The 
function keeps track of its position in the string sl between separat 
calls, so that subsequent calls made with the first argument an empty 
String will work through the string immediately following that token. 
this way subsequent calls will work through the string sl until no tokens 
remain. The separator string s2 may be different from call to call. 
When no token remains in sl, an empty value is echoed on stdout. 


Strstr echoes a substring starting at the first occurrence of string s2 in 
string Sil, eye notara aie GA (Cloves) iO (GXeleisue alin Gis Siew, Wil SA jo@ailimes 


TO 


JETA 


#:docstring strtrunc: 

wr Usages sercrune Sin Sud [992 S556] 

# 

# Used by many functions like strncmp to truncate arguments for comparison. 
# Echoes the first n characters of each string sl s2 on stdout. 

#:end docstring: 


22 
224 ###; 7; autoload 
22/5) IONE ILI Siewicicuinc (()) 


226 1 

229 mexSdL 8 xm 

228 forcipe 

229 echo welai 0k oma! 

230 done 

2315} 

232 

233 provide string 

234 

235 string.bash ends here 

236 

237 

238 

239 ==> Everything below here added by the document author. 
240 

241 ==> Suggested use of this script is to delete everything below here, 
242 ==> and "source" this file into your own scripts. 

243 

244 Siege dis 


245 string0-one 
246 stringl-two 


247 echo 

246 echo Vasitime W'smceue NU Functions Y 
249 echo "Original \"string0\" = $stringO" 
250 echo W\sieriing i \!Y = Sexual 

25 streat siue mri 

252 echo "New N"stringON" = $string0" 

253 echo 

254 

255 4 strlen 

256 echo 


257 eeno "eene; \Wsicielleia\/ friuarene sto rg Y 
258 str-123456789 

25) EEN UU Ex NU es Seer 

260 echo =m Viveingjicin oir \sice\ = U 

2491. gueielbew gtz 

262 echo 

263 

264 

265 

266 4 Exercise: 

BO {2 =a 

268 # Add code to test all the other string functions above. 
269 

270 

2741 exc 0 


Michael Zick's complex array example uses the md5sum check sum command to encode directory 
information. 


Example A-19. Directory information 


! /bin/bash 
directory-info.sh 
Parses and lists directory information. 


NOTE: Change lines 273 and 353 per "README" file. 


Michael Zick is the author of this script. 
Used here with his permission. 


(ee) A fey) (ab des (3. (S3) [l5 


Cont colis 

If overridden by command arguments, they must be in the order: 
Argl: "Descriptor Directory" 
Arg2: “Exclude Paths" 
Arg3: "Exclude Directories" 


Environment Settings override Defaults. 
Command arguments override Environment Settings. 


Default location for content addressed file descriptors. 
MD5UCFS-$(1:-S$(MD5UCFS:-'/tmpfs/ucfs'])) 


SS H H H H H H H EE 
O Wo) (es) ox Gal des GO [S d (€»» o) 


21 

22 Directory paths never to list or enter 

23 declare -a \ 

24 EXCLUDE PATHS-$(2:-S$(EXCLUDE PATHS:-'(/proc /dev /devfs /tmpfs)'}} 
25 

26 # Directories never to list or enter 

27 declare -a \ 

28 EXCLUDE DIRS-$(3:-S$(EXCLUDE DIRS:-'(ucfs lost+found tmp wtmp) '}} 
29 

30 # Files never to list or enter 

31 declare -a \ 

32 EXCLUDE FILES-$(3:-S$(EXCLUDE FILES:-'(core "Name with Spaces")']) 
33 

34 

25 Here document used as a comment block. 

36 ««LSfieldsDoc 

s 4 4 # List Filesystem Directory Information 4 4$ 4 # # 

38 

39 ListDirectory "FileGlob" "Field-Array-Name" 

40 or 

41 ListDirectory -of "FileGlob" "Field-Array-Filename" 

42 '-of' meaning 'output to filename' 

413 # # # # 

44 

45 String format description based on: ls (GNU fileutils) version 4.0.36 
46 

47 Produces a line (or more) formatted: 

48 inode permissions hard-links owner group 

4O SA NS. Ah 1 mszick mszick 

50 

51 size day month date hh:mm:ss year path 

52 215606008 Sun Aor 20 08559206 2003 /oareune msi Ck core 


53 

54 Unless it is formatted: 

55 inode permissions hard-links owner group 

A ATTS E RANE ——— JL root uucp 

57 

58 major minor day month date hh:mm:ss year path 
59 4, 68 Sun Apr 20 09:27:33 2003 /dev/ttyS4 

60 NOTE: that pesky comma after the major number 
61 

62 NOTE: the 'path' may be multiple fields: 


63 /home/mszick/core 

64 /proc/982/fd/0 -» /dev/null 

65 /proc/982/fd/1 -» /home/mszick/.xsession-errors 
66 /proc/982/fd/13 => /tmp/tmpfZVVOCs (deleted) 


oO) 
-1 


/proc/982/fd/7 -> /tmp/kde-mszick/ksycoca 


68 //incoxcw/ 992//o/G. => docket [ 1.3596] 

(SS). JOro SSZ TeS => ioes [11588] 

70 

71 If that isn't enough to keep your parser guessing, 


- 
N 


either or both of the path components may be relative: 
73 ../Burlt-Shared -» Built-Static 
FTA f l35900x—2 54 52). E206 lo“? => osf oci oc SIRCO/ Linowe- 2 «4 2) cae lowe 


76 The first character of the 11 (10?) character permissions field: 
TH "mg" Sexe 

73 "gl" Ipi1ceotoexy 

79 'b' Block device 

80 'c' Character device 

gl VI" feels aware 

82 NOTE: Hard links not marked - test for identical inode numbers 
83 on identical filesystems. 

84 All information about hard linked files are shared, xcept 

85 for the names and the name's location in the directory system. 
86 NOTE: A "Hard link" is known as a "File Alias" on some systems. 


87 '-' An undistingushed file 

88 

89 Followed by three groups of letters for: User, Group, Others 
90 Character 1: '-' Not readable; 'r' Readable 

Sil Character 2o Y= Nor yieatcelolkeo p \Wireskicalolle 

92 Character 3, User and Group: Combined execute and special 

93 '-' Not Executable, Not Special 


94 'x' Executable, Not Special 

95 's' Executable, Special 

96 'S' Not Executable, Special 

97 Character 3, Others: Combined execute and sticky (tacky?) 
98 '-' Not Executable, Not Tacky 

99 'x' Executable, Not Tacky 
100 't' Executable, Tacky 

101 'T' Not Executable, Tacky 


103 Followed by an access indicator 

104 Haven't tested this one, it may be the eleventh character 
105 or it may generate another field 

106 ' ' No alternate access 

107 '+' Alternate access 


108 LSfieldsDoc 

109 

110 

111 ListDirectory () 

ALZ 4 

LLS luexeuedl. =a T 

114 local -i of=0 # Default return in variable 
115 # OLD_IFS=SIFS # Using BASH default ' \t\n' 
116 

ol case "S#" in 

ia) E» Gee "SL atin 

que OE ) Quik 5. N [A 

JA LO returms do 

11211 GONE TE 

122 2) E # Poor man's "continue" 

12:5] 23) ieecuucig dh $5 

124 esac 

WAS) 

2G # NOTE: the (ls) command is NOT quoted (4) 

11223) T-( $(1s inod ignore-backups almost-all directory \ 
128 Tecos atq color-non time-status --sort-none \ 
129 --format-long $1) ) 

130 

jS case Sof in 

132 # Assign T back to the array whose name was passed as $2 
133 09) eva B2 NUNSSIDESIONINESU NE Eg 

134 # Write T into filename passed as $2 

W335 13 ecins USA OV x MSZ sa 

136 esac 

IS return 0 

138 } 

139 


140 # 4 # 4 # Is that string a legal number? # 4 # 4 # 


141 # 
142 # IsNumber "Var" 
143 4 4 4 4 # There has to be a better way, sigh... 


144 

145 IsNumber() 

146 ( 

147 Local =i Limit 

148 ai [ Si -—ee @ | 

149 then 

1250 return 1 

1151 else 

1:52 (let int-$1)  2»/dev/null 

153 return $? # Exit status of the let thread 
154 fest 

155. j 

156 

157 Fog od Index Filesystem Directory Information 4$ $4 # # # 
158 

15$ IndexList "Field-Array-Name" "Index-Array-Name" 

160 or 

161 IndexList -if Field-Array-Filename Index-Array-Nam 

162 IndexList -of Field-Array-Name Index-Array-Filenam 

163 IndexList -if -of Field-Array-Filename Index-Array-Filenam 
164 dog od 

165) 


166 : <<IndexListDoc 

167 Walk an array of directory fields produced by ListDirectory 

168 

169 Having suppressed the line breaks in an otherwise line oriented 
170 report, build an index to the array element which starts each line. 
dust 
172 Each line gets two index entries, the first element of each line 
173 (inode) and the element that holds the pathname of the file. 

174 

175 The first index entry pair (Line-Number--0) are informational: 


176 Index-Array-Name[0] : Number of "Lines" indexed 
177 Index-Array-Name[1] : "Current Line" pointer into Index-Array-Name 
178 


179 The following index pairs (if any) hold element indexes into 
180 the Field-Array-Name per: 


181 Index-Array-Name[Line-Number * 2] : The "inode" field element. 
182 NOTE: This distance may b Tener sli or +112 ESillememes 
183 Index-Array-Name[(Line-Number * 2) + 1] : The "pathname" element. 


184 NOTE: This distance may be a variable number of elements. 

185 Next line index pair for Line-Number+1. 

186 IndexListDoc 

187 

188 

199 

190 IndexList() 

i91 d 

1197] local, -a ELSI Local of listname passed 
qos local -a -i INDEX=( 0 0 ) Local of index to return 
194 load =i uab: TENE 
195 local -i if=0 of=0 Default to variable names 
196 
197 case "S4" in Simplistic option testing 
198 OJ sence d pp 
il 9j teburn i per 
200 fH Poor man's continue 
201. Caserma 

202 Sae mur XXE 

203 OR O= P 

204 No aehan aA 

205 exe » Gluut EB 

206 4) Shirl 2 qguelL fp Sadie. 9 Sloss 0 


(uo [Sy [— 


) 
) 
) 
) 


[SOF INS) N IST SY INS) ISS) INS) 


N 
ie) 


esac 
# Make local copy of list 
case "Sif" in 
0) eval LIST=\( \"\S\{S1\[@\]\}\" 
1) mrSr—( $ (cew $1) ) p7 
esac 
# Grok (grope?) the array 
Lent=${#LIST[@] } 
Lidx=0 
Troede CC iakeb< S= ibyesoe JY) 
do 
if IsNumber ${LIST[$Lidx] } 
then 
local -i inode name 
local ft 
inode=Lidx 
local m-S(LIST[S$Lidx-2]] 
IFES ILS IE E E A || Ss 150 
case $ft in 
b) ( (iskeber=i2))j) £9 
@)) (C(übsuober12)) BP 
*) ((Lidx+=11)) ;; 
esac 
name-Lidx 
Case Site aim 
22 ((Lidx+=1)) 7; 
b) ( (ue =1)) )) 5 
€) ((hicke=1))) 59 
d) ((huebe—1)) pF 
1) ((Lidx+=3)) ;; 
# A little mor legance her 
#+ sockets, deleted files later. 


# e 


# S{LIST[Sinode]} Name: 


cho 


else 


fa 
done 
case 


esac 
wee 


w aeee dL 


30) 
do 
( (Lidx+=1) ) 


Cory c 

INDEX [${#INDEX[*]}]=Sinode 

INDEX [${#INDEX[*]}]=Sname 

INDEX [0]=S{INDEX[0] }4+1 
"Line: S{INDEX[0]} Type: 
S${LIST [$name] }" 


( (Lidx+=1) ) 
Wee in 
0) eval $2=\( \"\S\{INDEX\[@\]\}\" 
1) sche VSD e] V9 S wea 
ici O 


, 


# 4$ 4 # Content Identify File # # # 4 4 


until IsNumber $(LIST[$Lidx]) 


fs CANTI I S 


, 


M RB 


+ 


Hard Links field 
meus SEE 


+ 


Block device 
Character device 
Anything else 


+ 


The easy one 

Block device 

Character device 

The other easy on 

At LEAST two more fields 


dE db db db dk 


would handle pipes, 


|) Crisis >= memg) } 


# Not required 


# One more "line" found 


$m Inode: \ 


X) 89 


# What could go wrong? 


DigestFile Input-Array-Name Digest-Array-Nam 
or 

DigestFile -if Input-FileName Digest-Array-Nam 
b GP uod 

Here document used as a comment block. 


(x GS) (CS) Wa} (ou (63) 193). (69) 


w w 
[SEE 
C Ko) 


««DigestFilesDoc 


The key (no pun intended) to a Unified Content File System (UCFS) 
is to distinguish the files in the system based on their content. 
Distinguishing files by their name is just, so, 20th Century. 


The content is distinguished by computing a checksum of that content. 
This version uses the md5sum program to generate a 128 bit checksum 
representative of the file's contents. 

There is a chance that two files having different content might 

g 

E 

c 


nerate the same checksum using md5sum (or any checksum). Should 
hat become a problem, then the use of md5sum can be replace by a 
yrptographic signature. But until then... 


The md5sum program is documented as outputting three fields (and it 
does), but when read it appears as two fields (array elements). This 
is caused by the lack of whitespace between the second and third field. 
So this function gropes the md5sum output and returns: 


[0] 32 character checksum in hexidecimal (UCFS filename) 

Lab] Single character: ' ' text file, '*' binary file 

[2] Filesystem (20th Century Style) name 

Note: That name may be the character '-' indicating STDIN read. 
DigestFilesDoc 


DigestFile() 
{ 


local if=0 # Default, variable name 
local =a TL 12 


case "S#" in 


3) case "S1" in 
-if) alg dL es SALA 6 °9 
ne) ieu JL E 
eee pA 
2) & SE # Poor man's "continue" 
ex) ien uc. dL Ep 
esac 


Gerster Sake aa 

0) wal mI WVS\iSl\ TEX VEN? X 
r= Seena SEW EI) | melssum =) ) 

1) 1"2e( SiwedSsum Sil) ) 


esac 


case S(4T2[0]) in 
Odor Cetin d^ 


1) weenciiem IL p 
2) Case eqr2|4]e0:2813 35m # SanScrit-2.0.5 
X93) 32] $4 802 (ey =o 102 [E1L 1] 8 10] 
312 [Lb ] eyes 


vr 


PYETA ee el P= See [LI] J 


TII i 
Pr; 
esac 
P; 
3) : ;; # Assume it worked 
Ss) serbe, dL pup 


esac 


local -i len-$(4T2[0]) 


399; 
340 
341 
342 
343 
344 
345 
346 
347 
348 
349 
S9 
Son 
352 
393 
354 
3395 
356 
Sb 
358 
S9 
360 
361 
362 
363 
364 
JOS 
366 
367 
368 
SOE, 
370 
373 
372 
33) 
374 
375 
376 
377 
378 
ST S) 
380 
381 
382 
383 
384 
385 
386 
387 
388 
39 
39 
391 
392 
393 
394 
395 
396 
399 
398 
399; 
400 
401 
402 
403 
404 


ait [ 


eval S2Q=\, ( 


sken -— 32 | 
VENSA (D2 HEN TAY 


, 


then return 1 


\) 


# # € # Locate File # # # # # 


Her 


<< SHE SNE 


Based on stat, 


ieldsDoc 


LocateFil [-1] FileNam 

or 

LocateFil [=i] exe tea lean 
dod: Sod 


version 2.2 


state incl State ile tud 


0 
1 


S) 


f= (=! Wer fee) <a) (ox; Gah dE Go 9) 


[12] 


[13] 


a d a. Per 
Return 
Size o 
Conten 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 
Elemen 


For a 
slat 
Scale = 


Stel = 
[0] 
[1] 


name 
Total size 


File - number of bytes 
Symbolic link - string length of pathname 


Number of 


(512 byte) 
File type and Access rights 


User ID of owner 
Group ID of owner 


Device number 
Inode number 


Number of hard links 


Device type 
Device type 


(if inode device) 
(if inode device) 


Time of last access 


May be disabled in 


'mount' 


PEE 


Location-Array-Name 


document used as a comment block. 


Location-Array-FileNam 


A file location is Filesystem-id and inode-number 


blocks allocated 


(hex) 


Major 
Minor 


with noatime 


utime, mknod 


atime of files changed by 
atime of directories changed by addition/deletion of files 


[Time of last modification 
mtime of files changed by write, 
mtime of directories changed by addtition/deletion of files 
Time of last change 
ctime reflects time of changed inode information 


xec, 


read, pipe, 


truncate, utime, mknod 


(owner, 


(link) information 


permissions, link count 
code: 0 
ie quETEEN S db 
ts of array 
t 0: /home/mszick 
[09/6 
NEED 
t 3: 41e8 
t 4s 500 
zr Se S900 
t Gs 303 
t PS: 9295959 
U eg 22 
c Ye © 
c digg 1 
t dig 30951221190 
t 128 OSTAIG 
t ISS 3951214966 
link in the form of linkname -> realname 
t linkname returns the linkname 
lt linkname returns the realname information 
EXE cuml Ee ken dead 
name 
IID=O 2 4 Maybe someday, 


but Linux stat structure 


(mmap?) 


group 


fis "ES Wis Es gEs fes ges ge gEs ges diy SS des. qfes quy qum. des es qus qus. des. Wes qiEy gius des (es Wes ew des Hus qiue qus. des. des fes ES gES. fes quy Qus. des des des qs. ges. Hus qux qux. pes des (Es qims des dus duy = Wess qus A E ques. ges. fus qv em. ges qus 
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Return 
Size o 
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PDR OR # does not have either LABEL nor UUID 
# fields, currently information must come 
# from file-system specific utilities 
will be munged into: 
UUID if possible 
Volume Label if possible 
"mount -l' does return the label and could return the UUID 


Maximum length of filenames 
Filesystem type 

Total blocks in the filesystem 
Free blocks 

Free blocks for non-root user(s) 
Block size of the filesystem 
Total inodes 

Free inodes 


code: 0 
i guexeN/ S Li 
ts of array 

0: /home/mszick 
Is 0 
2s 0 
Sue 255 
4: ef53 
OS LSA 
68 2277 LEO 
7: 2146050 
8: 4096 
9g ISLS S2 
Jg IA76425 
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StatFieldsDoc 


# Locat 


Fani [-1] FileName Location-Array-Name 


# Locat 


LocateFil 
{ 
toee 
Loesil 


case " 
OJ eer 
1) ret 
AA Bee 
*) whi 

do 


shi 
esac 


# More Sa 


L 


Li 


U 


Se SHE OE 


LOC= ( 


Fadi [-1] -of FileName Location-Array-FileNam 
&(0) 


-a LOC LOC1 LOC2 
Jesu ono 


S#" in 
vkan Ns 
rm 
; 


le (( WS m 2 ) 


case "$1" in 
EXE Dee I ease 
—OTIomtel s: 
s") daske dí op 
esac 
PE 
done ;; 


imsicuedE-2 s s 

OCil=( Sepe =e Silke Sil) j 

OC2e( Sar Ec SI S1) ) 

ncomment above two lines if system has "stat" command installed. 
${LOC1[@]:0:1} ${LOC1[@]:3:11} 

${LOC2[@]:1:2} ${LOC2[@]:4:1} ) 


(Gul Gal nl) Crh On). fen] yh Gai 


oul 
Nh f 
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case "Sof" in 
0) syal S2=\( VASALLO TEI NEN NX) 25 
4) ceo VS LUOS LEY x WeSZW px 
esac 
return 0 
Which yields (if you are lucky, and have "stat" installed) 
=S='5= LOSAIELOM DESCRITO coo 
Return code: 0 
Sime (Que qucxvewAS 5) 
Contents of array 
Element 0: /home/mszick 20th Century name 
Element 1: 41e8 Type and Permissions 
Element 2: 500 User 
Element 3: 500 Group 
Element 4: 303 Device 
Element 5: 32385 inode 
Element 6: 22 Link count 
Element 7: 0 Device Major 
Element 8: O0 Device Minor 
Element 9: 1051224608 Last Access 
Element 10: 1051214068 Last Modify 
Element 11: 1051214068 Last Status 
Element 12: 0 UUID (to be) 
Element 13: 0 Volume Label (to be) 
Element 14: ef53 Filesystem type 
} 
# And then there was some test code 
ListArray() # ListArray Name 
{ 
ocer =e Wei 
eval Ta=\( YUNSSNCSIESNIOSNI NTA" S) 
echo 
Sciigi v9 Se OIE Aay a 
eao WAS ous ariay Sls Serb iy 
echo "Contents of array sd" 
fone "(C a0 p aS poser gk Boobs jp) 
do 
cho Tttmileneacr Sas Snape] 
done 
return 0 
} 
declare -a CUR DIR 


For small arrays 
ListDirectory "S(PWD)" CUR DIR 
ListArray CUR DIR 


declare -a DIR DIG 
DigestFile CUR DIR DIR DIG 
echo "The new \"name\" (checksum) 


HON 


declare -a DIR ENT 
# BIG DIR # For 
# BIG-DIR # Lis 


declare -a DIR IDX 
# BIG-DIR # IndexList -if "/tmpfs/jun 
IndexList DIR ENT DIR, IDX 


declare -a IDX DIG 


${CUR_DIR[9]} is ${DIR_DIG[0]}" 


really big arrays - use a temporary file in ramdisk 
eDaliceroieroueyy ene WIS {CWI DOREL] 3e 58 
igneus MS (CUR DIR LIINI DIR 


N freman eu JOAN 
ENT 


k2" DIR IDX 


537 4 BIG-DIR £ DIR ENT-( $(cat /tmpfs/junk2) ) 

538 # BIG-DIR # DigestFile -if /tmpfs/junk2 IDX DIG 

539 DigestFile DIR ENT IDX DIG 

540 # Small (should) be able to parallize IndexList & DigestFile 

541 4 Large (should) be able to parallize IndexList & DigestFile & the assignment 
542 echo Ware rieme! (lex) oue che Comecsincs (ue GUUEID] as SID DIG O] i! 
543 

544 declare -a FILE LOC 
545 LocateFile ${PWD} FILE LOC 
546 ListArray FILE LOC 

547 

548 exit 0 


Stéphane Chazelas demonstrates object-oriented programming in a Bash script. 


Mariusz Gniazdowski contributed a hash library for use in scripts. 


Example A-20. Library of hash functions 


il Hash: 
2 Hash function library 
3 Author: Mariusz Gniazdowski «mariusz.gn-at-gmail.com-» 
4 Date: 2005-04-07 
3 
6 Functions making emulating hashes in Bash a little less painful. 
7 
8 
9 Limitations: 
1O * Only global variables are supported. 
at * Each hash instance generates one global variable per value. 
12 * Variable names collisions are possible 
13 #+ if you define variable like _ hash_ hashname_key 
14 * Keys must use chars that can be part of a Bash variable name 
15 #+ (no dashes, periods, etc.). 
16 * The hash is created as a variable: 
Ly hashname_keyname 
18 So if somone will create hashes lik 
Lg myhash_ + mykey = myhash__mykey 
20 myhash + _mykey = myhash__mykey 
21 Then there will be a collision. 
22 (This should not pose a major problem.) 
29 
24 
25 Hash config varname prefix- hash . 
26 
AT 
28 Emulates:  hash[key]-value 
Params: 
il = basm 
2 — key 
3 —- value 


function hash set { 
eval "$(Hash config varname prefix)]$(1) $(2)-N"$(3)N"" 


CO CO CO CO CO CO CO CO CO CO Dd 
We) (geh SS] Oy Gl ges (3) hy [5 99» (o) 


# Emulates:  value-hash[key] 
40 # 
41 # Params: 
42 # 1 - hash 
43 # 2 - key 
44 # 3 —- value (name of global variable to set) 


45 
46 
47 
48 
49 
50 
Sl 
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84 
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89 
90 
91 
92 
93 
94 
95 
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2) 
98 
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100 
101 
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103 
104 
105 
106 
107 
108 
109 
110 


function hash get into { 
Swell WS sw SIS sreusile. Coni al (o]- vannene Oraris Sl 9 12 EN UP 
} 


# Emulates: echo hash[key] 

# 

# Params: 

# 1 - hash 

# 2 - key 

# 3 — echo params (like -n, for example) 


function hash_echo { 
eval "echo $3 \"\$S${Hash_config_varname_prefix}${1}_${2}\"" 


# Emulates: hashl[keyl]=hash2[key2] 
# 

# Params: 

# 1 - hasht 

# 2 - keyl 

# 3 - hash2 

# 4 - key2 


Tineti Masia copy 1 

eval "S{Hash_config_varname_prefix}${1}_${2}\ 
=\"\$S${Hash_config_varname_prefix}${3}_S${4}\"" 
} 


Emulates: hash[keyN-1]=hash[key2]=...hash[key1] 


Copies first key to rest of keys. 


Params: 

i = hae 
2 - keyl 
S — key 
N - keyN 


function hash dup { 

local hashName="$1" keyName-"$2" 

Slane 2 

wawil | Ss e Ir co 

eval "$(Hash config varname prefixj$(hashName) $(1)^ 

=\"\SS${Hash_config_varname_prefix}${hashName}_S${keyName}\"" 

Saal irie c 

done; 


Emulates: unset hash[key] 


dL c devenu 
2 - key 
function hash unset { 
eval "unset $(Hash config varname prefix)$(1) $(2)]" 


# 
Ls 
# Params: 
# 
# 


Emulates something similar to:  ref-&hash[key] 
The reference is name of the variable in which value is held. 


Params: 
ib c daveisini 


Se oce che ch cod 
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# 2 - key 
# 3 - ref - Name of global variable to set. 
function hash get ref into { 
eval "$3-N"S$(Hash config varname prefix)$(1) $(2)N"" 

} 

Emulates something similar to: echo &hash[key] 

That reference is name of variable in which value is held. 

Params: 

dL — imeem 

= AEN) 

3 - echo params (lik n for example) 
function hash_echo_ref { 

eval Yecha SS USD. comic varcnans qecrexue bo SL} Sip ww 

} 
# Emulates something similar to:  $$hash[key] (paraml, param2, 
# 
# Params: 
# 1 — hash 
# 2 - key 
# 3,4, ... — Function parameters 


function hash_call { 
local hash key 
hash=$1 
key=$2 
Slaiiric 2 


eval "eval \"\$${Hash_config_varname_prefix}${hash}_${key} \\\"\\\S@\\\"\"n 


Emulates something similar to:  isset(hash[key]) or hash[key]==NULL 
Params: 
iL c NASA 
2 — key 
Returns: 
0 - there is such key 
1 - there is no such key 
Function hash is set { 
eval "if [[ \"\${${Hash_config_varname_prefix}${1}_${2}-a}\" = N"aN" && 
WON SHES TUS te vetat eon sey Ayeliciovelniey premiki iL 9102) Soh m Voy qT 


then return 1; else return 0; fi" 


Emulates something similar to: 
foreach(Shash as Skey => Svalue) ( fun($key,$value); } 


Params: 
dL <= mein 
2) c iGIACIE ae Joyce 


function hash foreach { 
local keyname oldIFS="SIFS" 
IFS=' ' 


for i in $(eval "echo N$(!$(Hash config varname prefix)$(1) *)"); 
keyname-$ (eval "echo WM$(if4$(Hash config varname prefix)$(1) )") 


eval "$2 Skeyname \"\$$i\"" 


It is possible to write different variations of this function. 
Here we use a function call to make it as "generic" as possible. 


do 


TW done 

ITS ASW SOATGA 

i179 N 

180 

181 # NOTE: In lines 103 and 116, ampersand changed. 

182 # But, it doesn't matter, because these are comment lines anyhow. 


Here is an example script using the foregoing hash library. 


Example A-21. Colorizing text using hash functions 


1 #!/bin/bash 
2 # hash-example.sh: Colorizing text. 
3 # Author: Mariusz Gniazdowski «mariusz.gn-at-gmail.com» 


4 
5 Hash.lib # Load the library of functions. 
6 
7 hash set colors red AOSS [Oe Shiba! 
8 hash_set colors blue IN DSS Oe Sain” 
9 hash_set colors light_blue uA OSS ie Bahan 
10 hash set colors light red TN (0133533 Ede Sibi 
11 hash set colors cyan AOSS O ome 
l2 Masa Sere Colors licii creen AOSS eS 
13 hash, set colors light, gray UNOS3: Oe Bay 
14 hash_set colors green DN OSS (Ole Sig 
15 hash_set colors yellow UNOS} I de $k Shen!!! 
16 hash set colors istic _jowueoile AOSS Le sisi! 
17 hash_set colors purple TN (033 35/0 eS Sian! 
JLS. hash ser colores reset color “Y\WORS 0" 
1g 
20 
Zi $1 — keyname 
22) $2 — value 
23 ery colors 4 
24 Scho er YSZ" 
25 exco: Wiring: shane: als: Sil 
26 h 
AW oaea 3areyeexeKcim Calore teres Colors 


N 
[99] 


hash echo colors reset color -en 


echo -e '\nLet us overwrite some colors with yellow.\n' 

# It's hard to read yellow text on some terminals. 

hash_dup colors yellow red light_green blue green light_gray cyan 
hash_foreach colors try_colors 

hash echo colors reset color -en 


cho Hotet us deletes then emei try Colors once mone g o BU 


for i in red light green blue green light gray cyan; do 
hash unset colors $i 

done 
h foreach colors try colors 
hash echo colors reset color -en 


CO CO CO CO CO CO CO CO CO CO DN 
oOo COO OP O9 NS ESSO XO 


Po 
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2) 
w 
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hash set other txt "Other examples . . ." 
hash echo other txt 

h get into other txt text 

echo Stext 


hash set other my fun try colors 
hash call other my fun purple "'hash echo colors purple'" 
h echo colors reset color -en 


Or Pe am am am a4m aum 
(ex Wo. (os) =] (ex, (On des feb fno) 
i 
w 
[0] 
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echo; echo "Back to normal?"; echo 


evade SP 


On some terminals, the "light" colors print in bold, 
and end up looking darker than the normal ones. 
Why is this? 


An example illustrating the mechanics of hashing, but from a different point of view. 


Example A-22. More on hash functions 


NPRPRPRPRP PPP PY 
O io 00 -1 O Qi i (). M HP. O xo 0 - O O1 4 CQ Io E 
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41 


OP gu bb SP SP aum 
«5» Wer (oo. =] (x, Gr dex Go [nos 


! /bin/bash 
rcis ba Slaw 1.2 2005/04/21 299249219 doles ix © 
Copyright 2005 Oliver Beckstein 
Released under the GNU Public License 
Author of script granted permission for inclusion in ABS Guide. 
(Thank you!) 


pseudo hash based on indirect parameter expansion 
API: access through functions: 


create the hash: 


newhash Lovers 


add entries (note single quotes for spaces) 


addhash Lovers Tristan Isolde 
addhash Lovers 'Romeo Montague' 'Juliet Capulet' 


access value by key 
gethash Lovers Tristan ====> oleis 
show all keys 
keyshash Lovers c———3À Viiensitaim! 'Romeo Montague' 
Convention: instead of perls' foo{bar} = boing' syntax, 
use 
' foo bar-boing' (two underscores, no spaces) 
1) store key in | NAME keys[] 


2) store value in NAME values[] using the same integer index 
The integer index for the last entry is | NAME ptr 


NOTE: No error or sanity checks, just bare bones. 


EUMCIEROM - zuepideveusim ( 4 


# private function 

# call at the beginning of each procedure 
# defines: keys values | ptr 

# 

# Usage: _inihash NAM 
local name=$1 
_keys=_${name}_keys 
_values=_${name}_values 
_ptr=_${name}_ptr 
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} 


function newhash () { 
# Usage: newhash NAME 
# NAME should not contain spaces or dots. 


# Actually: it must be a legal name for a Bash variable. 


# We rely on Bash automatically recognising arrays. 


local name=$1 

local _keys _values _ptr 
_inihash ${name} 

eval ${_ptr}=0 


function addhash () { 
# Usage: addhash NAME KEY 'VALUE with spaces' 


# arguments with spaces need to be quoted with single quotes 


IkOCAIL mamMeSSil kHEVSQ ugsu 
local _keys _values _ptr 
.inihash ${name} 


#echo "DEBUG(addhash): ${_ptr}=${!_ptr}" 
eval let S{_ptr}=S{_ptr}+1 


eval US keysls il oer} ISVS te ym 
evel UIS yvaluss Sil e) yg] UM 


function gethash () ( 
# Usage: gethash NAME KEY 
# Returns boing 
d ERR-0 if entry found, 1 otherwise 
j^  JimenE "S not eL jowcojoewe hasa c 
#+ we simply linearly search through the keys. 
local name-$1 key="$2" 
local keys values | ptr 
local k y a. toting! 1m 
_inihash ${name} 


_ptr holds the highest index in the hash 
found=0 


Fore 3L iia S (Geexer il Stl joe) -p clo 


h="\S{${_keys} [S{i}]}" 4$ Safer to do it in two steps, 


eval k-$(h) #+ especially when quoting for spaces. 


Alc 1 Saka 
done; 


"S(key)" ]; then found=1; break; fi 


${found} 0 Mise recura dp 

else: i is the index that matches the key 
oS NSS sedes) [qat p p v 

eval echo "S{h}" 

recura 07 


function keyshash () { 
4 Usage: keyshash NAME 


# Returns list of all keys defined for hash name. 


local name-$1 key="$2" 
local keys values ptr 
«oca ice 

_inihash ${name} 


# _ptr holds the highest index in the hash 
for i dim S(eesg 1 Sil ptrj)s do 
h="\S${${_keys} [${i}]}" i Sede co Ck ie aim CwO 


steps 


WN 
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1122) 
1,223] 
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126 
Wg 
128 
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130 
Sak 
L32 
13.9) 
134 
LSS) 
136 
1553 
138 
1,99; 
140 
141 
142 
143 
144 


) 


# 


# 
# 


eval k-$(h) #+ especially when quoting for spaces. 
echo -n "'$(k)' ™ 
done; 


Joys dieu m CESSE ME; 
(Per comments at the beginning of the script.) 


newhash Lovers 


addhash Lovers Tristan Isolde 


addhash Lovers 'Romeo Montague' 'Juliet Capulet' 

# Output results. 

echo 

gethash Lovers Tristan # Isolde 

echo 

keyshash Lovers # 'Tristan' 'Romeo Montague' 


echo; echo 


exit 0 


Exercise: 


Add error checks to the functions. 


Now for a script that installs and mounts those cute USB keychain solid-state "hard drives." 


Example A-23. Mounting USB keychain storage devices 


Mop pop ppppmpogÀ: 
O io 0» -1 O Oi i (). M P. O x0 0 -1 O O1 4 CQ I 5 
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!/bin/bash 

==> usb.sh 

==> Script for mounting and installing pen/keychain USB storage devices. 
==> Runs as root at system startup (see below). 


==> Newer Linux distros (2004 or later) autodetect 
==> and install USB pen drives, and therefore don't need this script. 


E Buh, abe (s SidsLilil sasieicuyeic iw, 


This code is free software covered by GNU GPL license version 2 or above. 
Please refer to http://www.gnu.org/ for the full license text. 


Some code lifted from usb-mount by Michael Hamilton's usb-mount (LGPL) 


+ see http://users.actrix.co.nz/michael/usbmount.html 


INSTALL 


Put this in /etc/hotplug/usb/diskonkey. 
Then look in /etc/hotplug/usb.distmap, and copy all usb-storag ntries 


+ into /etc/hotplug/usb.usermap, substituting "usb-storage" for "diskonkey". 


Otherwise this code is only run during the kernel module invocation/removal 


+ (at least in my tests), which defeats the purpose. 


Handle more than one diskonkey device at one time (e.g. /dev/diskonkeyl 


+ and /mnt/diskonkeyl), etc. The biggest problem here is the handling in 
+ devlabel, which I haven't yet tried. 


AUTHOR and SUPPORT 
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Konstantin Riabitsev, «icon linux duke edu». 
Send any problem reports to my email address at the moment. 


Se oc che cb db 


==> Comments added by ABS Guide author. 


SYMLINKDEV-/dev/diskonkey 
MOUNTPOINT-/mnt/diskonkey 
DEVLABEL-/sbin/devlabel 
DEVLABELCONFIG-/etc/sysconfig/devlabel 
IAM-$0 


## 
# Functions lifted near-verbatim from usb-mount code. 
# 
function allAttachedScsiUsb { 
giae foro sesi/ sata Toros cietsi usb-storage ise dc. | 
xargs grep -l1 'Attached: Yes' 
} 
function scsiDevFromScsiUsb { 
echo $1 | awk -E"[-/]" '{ n=S(NF-1); 
print "/dev/sd" substr("abcdefghijklmnopqrstuvwxyz", ntl, 1) 


sae [p VE TACTIIONTA = Vachs | me [ Sie USWDEWVICS o Taan 

## 

# lifted from usbcam code. 

# 

zug [p ed Joweuc/acwuad Come lle, leek jp Toen 
CONSOLEOWNER= cat /var/run/console.lock' 

else | -~i /yew/ lock /eomsole, lock je imeem 
CONSOLEOWNER-' cat /var/lock/console.lock'^ 


else 


CONSOLEOWNER- 


ial 

for procEntry in $(allAttachedScsiUsb); do 
scsiDev=$ (scsiDevFromScsiUsb SprocEntry) 

Some bug with usb-storage? 


accessed 


Partitions ersa not in (joe /Oeucie aie alors} mebl they Ar 
+ somehow. 
/sbin/fdisk -1 $scsiDev >/dev/null 


ah ab. Wises TE ntire device and hopes for the better. 


if grep -q 'basename $scsiDev'1 /proc/partitions; then 
part="$scsiDev""1" 

else 
part=SscsiDev 

if ab 

## 


Most devices have partitioning info, so the data would be on 
+ /dev/sd?1. However, some stupider ones don't have any partitioning 
+ and use th ntire device for data storage. This tries to 

+ guess semi-intelligently if we have a /dev/sd?1 and if not, then 


# Change ownership of the partition to the console user so they can 


#+ mount it. 

# 

su [| d = “SICONSIOIMIO MINI TB. xls 
chown SCONSOLEOWNER:disk Spart 


I 


ita 
Td 


# This checks if we already have this UUID defined with devlabel. 


n Jie ioe, sic ien ecke icine clewalee co were. Met, 
# 


9 prodid-' SDEVLABEL printid -d $part' 
98 if ! grep -q Sprodid SDEVLABELCONFIG; then 
2 # cross our fingers and hope it works 
100 SDEVLABEL add -d $part -s $SYMLINKDEV 2>/dev/null 
TOIL fi 
LOZ ## 
119) # Check if the mount point exists and create if it doesn't. 
104 # 
105 if [ ! -e SMOUNTPOINT ]; then 
106 mkdir -p SMOUNTPOINT 
107 aca 
108 ## 
109 # Take care of /etc/fstab so mounting is easy. 
110 # 
i33 if ! grep -q "^$SYMLINKDEV" /etc/fstab; then 
ae # Add an fstab entry 
1.1.3 echo -e \ 
114 "SSYMLINKDEV\t\tS$MOUNTPOINT\t\tauto\tnoauto, owner,kudzu 0 0" \ 
115 >> /etc/fstab 
116 3E 3. 
TEE) done 
118 xd (qp P - Wem Ja Ehem 
i ## 
120 # Make sure this script is triggered on device removal. 
ut # 
11277 mkdir -p “dirname S$REMOVER' 
12:9 ln -s $IAM SREMOVER 
124 3E dL 
125 elif [ "S{ACTION}" = "remove" ]; then 
126 ## 
3:239) # If the device is mounted, unmount it cleanly. 
128 # 
129 if grep -q "SMOUNTPOINT" /etc/mtab; then 
130 # unmount cleanly 
1951 umount -1 $MOUNTPOINT 
SZ, iE dL 
1553) ## 
134 # Remove it from /etc/fstab if it's there. 
135 # 
136 if grep -q "^$SYMLINKDEV" /etc/fstab; then 
137 grep -v "^$SYMLINKDEV" /etc/fstab > /etc/.fstab.new 
138 mv -f /etc/.fstab.new /etc/fstab 
13$ iE aL 
AYO seal 
141 
142 exit 0 


Converting a text file to HTML format. 


Example A-24. Converting to HTML 


1 #!/bin/bash 

2 tohtml.sh [v. 0.2, reldate: 06/26/08, still buggy] 

3 

4 Convert a text file to HTML format. 

5 Author: Mendel Cooper 

6 License: GPL3 

7 Usage: sh tohtml.sh « textfile » htmlfile 

8 Script can easily be modified to accept source and target filenames. 
9 
10 Assumptions: 
dit 1) Paragraphs in (target) text file are separated by a blank line. 
1.2 2) Jpeg images (*.jpg) are located in "images" subdirectory. 
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In the target file, the image names ar nclosed in square brackets, 
[image01. jpg]. 
3) Emphasized (italic) phrases begin with a space+underscore 

or the first character on the line is an underscore, 

and end with an underscore+space or underscoretend-of-lin 


for example, 


Settings 
TSIZE-2 


IMGDIR="images" 


Headers 
RO1='<!DOCTYPE 


# Small-medium font size 
# Image directory 


HTML PUBLIC "-//W3C//DID HTML 4.01 Transitional//EN"»' 


102'«html»' 
|1-'«head»' 
Lie Ue leere " 
12a='<title>' 
12b='</title>' 


R022'«! Conyac eel 1o» RUUMI low; *""EgiENNL Sins Gerri ==>! 
R032'«!-- script author: M. Leo Cooper <thegrendel.abs@gmail.com> --»' 


121='<META NAM 


E-"G 


ENERATOR" CONTENT-"tohtml.sh script">' 


14bp-2'»' 
OGULOIÍS 
10='</body>' 
11='</html>' 


LD="<b>" 


CE 


r 


NTER="<center>" 


" 


EI 


Wr 


pr 


="<þr>" 


ite_headers () 
{ 

echo "SHDRO1" 
echo 
echo "SHDRO2" 
echo "SHDRO3" 
echo 
echo 
echo "SHDR10" 
echo "SHDR11" 
echo "SHDR121" 
echo "SHDRila" 
echo "SHDR13" 
echo 

excl =m, VARDERA 
eelne sin SIINPU SU 
echo "SHDR14b" 


echo "SBOLD" 


ocess text () 


{ 


while read line 
do 
ie [|  wenbue 


echo "SLF" 
echo "SLF" 


e 
ZE 


] 


13='<body bgcolor="#dddddd">' # Change background color to suit. 
14a='<font size=' 


D_CENTER="</center>" 


# Everything in bold (more easily readable). 


# Read one line at a time. 


# Blank line? 
# Then new paragraph must follow. 


# Insert two <br> tags. 


78 continue # Skip the underscore test. 


80 else # Otherwise 

81 

82 as [[ "Sae" == "Al egay II a XS a grapmie? 
83 then # Strip away brackets. 
84 tewuqseS( echo "SIxme" | sed = "s/XI//" =e "s/XLZZV y 
85 line=""SCENTER" «img src="\"SIMGDIR"/Stemp\"> "SEND CENTER" " 
86 # Add image tag. 
87 # And, center it. 
88 E 3 

89 

90 iE dL 

91 

92 

93 echo YSlinet | gred «p — 

Sja me T "Sew eee Q J # If line contains underscore 
95 then 

96 # 

97 # Convert underscored phrase to italics. 

98 temp-$( echo "Sline" | 

99 Geol m m f/f cu" Um view wo] 
100 Ged| =e Uef/A_feisy/! =e "m7 feN/A3e79 J 
101 # Process only underscores prefixed by space, 
102 #+ or at beginning or end of line. 
103 # Do not convert underscores embedded within a word! 
104 line="Stemp" 
105 # Slows script execution. Can be optimized? 
106 # 
LOF deal 
108 
109 
LiL) 
iL abil echo 
1327 echo "$line" 
ILS echo 
114 ) # End while 
RS) done 
IL } # End process_text () 
TE) 
TER 
119 write footers () # Termination tags. 
120 { 


1231 echo "SFTR10" 
122 echo "SFTR11" 


128 write_headers 
129 process_text 
130 write footers 


138 # 1) Fixup: Check for closing underscore before a comma or period. 
139 # 2) Add a test for the presence of a closing underscore 
140 #+ in phrases to be italicized. 


Here is something to warm the hearts of webmasters and mistresses: a script that saves weblogs. 


Example A-25. Preserving weblogs 


Mop p opp ppppÀDGÀO:! 
O (o 00 -1 O Oi i$ (). NM P. O xo 0 -1 O O1 i CQ Io 
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U PP Pe SP um aum aum 
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[91 
N 


59 


(on ah ml 
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57 


y O O TT 
OEE Wey tool 


63 
64 


! /bin/bash 
archiveweblogs.sh v1.0 


Used with permission. 


PROBLEM-66 


LOG DAYS-"4 3 2 1" 
LOG DIR-/var/log/httpd 


Default Apache/RedHat stuff 


Troy Engel <tengel@fluid.com> 
Slightly modified by document author. 


Set this to your backup dir. 
BKP_DIR=/opt/backups/weblogs 


,0OG_FILES="access_log error log" 
Default RedHat program locations 

LS-/bin/ls 

MV-/bin/mv 

ID=/usr/bin/id 


CUT=/bin/cut 


This script will preserve the normally rotated and 
+ thrown away weblogs from a default RedHat/Apache installation. 
It will save the files with a date/time stamp in the filename, 
+ bzipped, to a given directory. 


Run this from crontab nightly at an off hour, 
+ as bzip2 can suck up some serious CPU on huge logs: 
02 * * * /opt/sbin/archiveweblogs.sh 


echo "PANIC: SBKP_DIR doesn't exist or isn't writable!" 


COL=/usr/bin/column 

BZ2=/usr/bin/bzip2 

# Are we root? 

USER= $ID -u' 

ase || “esis IS Ug jp spem 
exem "ESOS Only ise Cam iin lake Guess LA 
exit SPROBLEM 

TE aL 

# Backup dir exists/writable? 

3e qp d = SIME Dis jp then 
exit SPROBLEM 

FI 

# Move, rename and bzip2 the logs 

for logday in S$LOG DAYS; do 


for logfile in SLOG FILES 


, 


do 


MYFILE-"$LOG DIR/$1ogfile.$10gday" 


Tf [| -w SMYETLE ]; then 


DTS-'$LS -1go time-styl 


T$Y$m$d SMYFILE | $COL -t | $CUT 


SMV SMYFILE SBK? DIR/$1logfile.$DTS 
$BZ2 SBKP. DIR/S$1logfile.S$DTS 

else 
# Only spew an error if the fil xits (ergo non-writable). 
if | —f SMYFILE ]; then 

echo "ERROR: SMYFILE not writable. Skipping." 

fi 

if a 

done 
done 


65 
66 e 
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How to 


keep the shell from expanding and reinterpreting text strings. 


Example A-26. Protecting literal strings 


Se OH 


# 


NPRPRPPRP PPP PY 
SCOMIDTUHKRWNHHFOWOMAIDGURWNE 


[S9 ISS) DOP TROY ISP ISS) [55 
SI] xexy Gl dev ea Is) [ 


N 
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! /bin/bash 
protect_literal.sh 


set -vx 


gasi Protect lesiel Sire Doe 


Copyright (c) Michael S. Zick, 2003; All Rights Reserved 
License: Unrestricted reuse in any form, for any purpose. 
Warranty: None 
Revision: $ID$ 


Documentation redirected to the Bash no-operation. 
Bash will '/dev/null' this block when the script is first read. 
(Uncomment the above set command to see this action.) 


Remove the first (Sha-Bang) line when sourcing this as a library 
procedure. Also comment out th xample use code in the two 
places where shown. 


USqe: 
.protect literal str 'Whatever string meets your S${fancy}' 
Just echos the argument to standard out, hard quotes 
restored. 


$( protect literal str 'Whatever string meets your S$(fancy)') 
as the right-hand-side of an assignment statement. 


Does: 
As the right-hand-side of an assignment, preserves th 
hard quotes protecting the contents of the literal during 


assignment. 

Notes: 
The strange names (_*) are used to avoid trampling on 
the user's chosen names when this is sourced as a 
library. 


40 _Protect_Literal_String_Doc 


[i 
[ER 


42 # 


Bs 
Co 


Mave, eec LUSE eene EAE O Eoi 


44 protect literal str() ( 


# 
# 


U uS PP 
(Sy Mor (oe) 1) yy Gal 


# 


(Gm (Gal (nh Orb (Sm (Oni 
(ex, (rl dE (Sh [R5 [13 


Pick an un-used, non-printing character as local IFS. 
Not required, but shows that we are ignoring it. 
local IFS-$'Nx1B' # NESC character 
Enclose the All-Elements-Of in hard quotes during assignment. 
local tmp=$'\x27'S$@$'\x27' 
local deem N USES UY # Even uglier. 
local len=${#tmp} # Info only. 
echo $tmp is $len long. # Output AND information. 


59 
59 


ererererrrrrre 


m 
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# This is the short-named version. 


-jee( 
LOCAL 309 ems esL" # NESC character (not required) 
Selo So WO VEHI Vos gU # Hard quoted parameter glob 


i? gv PrO Ct Jine. SET Wests Y 
# # Remove the above "t " to disable this code. # # # 


# See how that looks when printed. 


(Gag, Ul Test On y 
protect lireral_ str kello Susser" 
T protect literal str 'Hello "S{username}"' 
echo 


Which yields: 
Test On 
"Hello Suser' is 13 long. 
"Hello "$(username)"' is 21 long. 


Looks as expected, but why all of the trouble? 

The difference is hidden inside the Bash internal order 
+ of operations. 

Which shows when you use it on the RHS of an assignment. 


Declare an array for test values. 
declare -a arrayZ 


Assign elements with various types of quotes and escapes. 
euereN MES zero "S els "helle Sms") "ello Ses" U\ Passe Spp 7 


Now list that array and see what is there. 
echot a MDC IWON Y 

ior (( xeX) p S hiana Ale p do 3) 

do 


echo Element $i: S${arrayZ[$i]} is: ${#arrayZ[$i]} long. 
done 


echo 
Which yields: 
D a ee e LE 
Element 0: zero is: 4 long. # Our marker element 
menene ie "ellc; HANE ales 1.98 Tome. t Que VS le "5.59 JY 
Element 2: Hello ${You} is: 12 long. # Quotes are missing 
mike Se Eassa WU shes 10 Toner 4 S{pw} expanded to nothing 
Now make an assignment with that result. 
declare -a array2-( S${arrayZ[@]} ) 
And print what happened. 
choni Test Thr W 
for (( aeg p. sS e g axe ) 
do 
echo Element $i: S{array2[$i]} is: ${#array2[$i]} long. 
done 
echo 
# Which yields: 
# Test Thr 
# Element 0: zero is: 4 long. # Our marker element. 
# Element 1: Hello ${Me} is: 11 long. # Intended result. 
# Element 2: Hello is: 5 long. # S(You) expanded to nothing. 
# Element 3: 'Pass: is: 6 long. # Split on the whitespace. 
i? iengo “ie y aeg E ones # The end quote is here now. 


) 


12:3) 


124 Our Element 1 has had its leading and trailing hard quotes stripped. 
112255) Although not shown, leading and trailing whitespace is also stripped. 
126 Now that the string contents are set, Bash will always, internally, 
127 #+ hard quote the contents as required during its operations. 

128 

129 Why? 

iL SO) Congicering our "S ole kello Sj") V qoos o 

1331 Jo eg | c Jrqeucehheu. iecxonuonxerexelo Bice icine GMOESS - 

dL S2 S( se ) => deles waitin ipe wesguilt. Qits..so GENES clus. 

1133 wjolle "U Ges V =} @allileel wuadEla licere enceten; STAs CNS Cuore. 
134 The result returned includes hard quotes; BUT the above processing 
135 #+ has already been done, so they become part of the value assigned. 
136 

29r] Similarly, during further usage of the string variable, the ${Me} 
138 #+ is part of the contents (result) and survives any operations 

139 (Until explicitly told to evaluate the string). 

140 

141 Hint: See what happens when the hard quotes ($'\x27') are replaced 
142 #+ with soft quotes ($'\x22') in the above procedures. 

143 Interesting also is to remove the addition of any quoting. 

144 

1415 .Protect Literal String Test 

146 # 4 Remove the above "# " to disable this code. 4 # # 

147 

148 exit 0 


But, what if you want the shell to expand and reinterpret strings? 


Example A-27. Unprotecting literal strings 


! /bin/bash 
unprotect literal.sh 


-Heock 


# set -vx 
pee qheieseonEcetE alites fSiExcaLimier JDIexer" 


Copyright (c) Michael S. Zick, 2003; All Rights Reserved 
License: Unrestricted reuse in any form, for any purpose. 
Warranty: None 
Revision: $ID$ 


Documentation redirected to the Bash no-operation. Bash will 
! ve tes oull cns Toylcoyelle vaen che Sielichijoye Le Eiro roae 
(Uncomment the above set command to see this action.) 


Remove the first (Sha-Bang) line when sourcing this as a library 
procedure. Also comment out th xample use code in the two 
places where shown. 


NPRPRPPRP PPP PY 
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DSgue 
Complement of the "S$( pls 'Literal String')" function. 
(S the protect_literal.sh example.) 


N 
[99 


NON 
nl (ss 


StringVar-$( upls ProtectedSringVariable) 


No ON 
-1 Oo 


N 
[99] 


Does: 
When used on the right-hand-side of an assignment statement; 
makes the substitions embedded in the protected string. 


(G3) (69) (69v |S} 
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Notes 4 
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The strange names (_*) are used to avoid trampling on 
the user's chosen names when this is sourced as a 
library. 
_UnProtect_Literal_String_Doc 
—upls() { 
lkexexedl 109 Ss see! # NESC character (not required) 
eval echo S@ # Substitution on the glob. 
} 
u^ S" lepore. Literal Serine Test 
# 4 # Remove the above "f£ " to disable this code. # # 4 
als) 4 
LoCall 309 Sys sc lis 4 NESC character (not required) 
echo S27) SESY Veg" # Hard quoted parameter glob 
} 
Declare an array for test values. 
declare -a arrayZ 
Assign elements with various types of quotes and escapes. 
EucrewUEES( zero "S pls "eligo Sdque)w Mello Sow" UW\ Pesce Simpy Y 
Now make an assignment with that result. 
declare -a array2-( ${arrayZ[@]} ) 
Which yielded: 
Test Thr 
Element 0: zero is: 4 long # Our marker element. 
Element 1: Hello ${Me} is: 11 long # Intended result. 
Element 2: Hello is: 5 long 4 S(You) expanded to nothing. 
Jewel. Sg “Poses aeg © Ju # Split on the whitespace. 
Hihewwewwr Aa " des Jb Long # The end quote is here now. 
EON SE 
Initialize 'Me' to something for th mbedded $(Me) substitution. 
This needs to be done ONLY just prior to evaluating the 
* protected string. 
(This is why it was protected to begin with.) 
Me-"to the array guy." 


# Set a string variable destination to the result. 
newVar-$( upls S${array2[1]}) 


# Show what the contents ar 
echo $newVar 


# Do we really need a function to do this? 
newerVar=S (eval echo S{array2[1]}) 
echo $newerVar 


# I guess not, but the  upls function gives us a place to hang 
#+ the documentation on. 

# This helps when we forget what a # construction like: 

#+ S(eval echo ... ) means. 


# What if Me isn't set when the protected string is evaluated? 
unset Me 

newestVar-$( upls $(array2[1])) 

echo $newestVar 


VOO Just gone, no hints, no runs, no errors. 

101 

102 Why in the world? 

LOS Setting the contents of a string variable containing character 
104 #+ sequences that have a meaning in Bash is a general problem in 
105 #+ script programming. 

106 

TOT This problem is now solved in eight lines of code 

108 #+ (and four pages of description). 

109 

ALO) Where is all this going? 

PLIL Dynamic content Web pages as an array of Bash strings. 

112 Content set per request by a Bash 'eval' command 

113 #+ on the stored page templat 

114 ot intended to replace PHP, just an interesting thing to do. 
1.115 # 

EG Don't have a webserver application? 

TEE) o problem, check th xample directory of the Bash source; 
118 #+ there is a Bash script for that also. 

119 

120 .UnProtect Literal String Test 

121 # 4 Remove the above "# " to disable this code. 4 # # 

15272 
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This interesting script helps hunt down spammers. 


Example A-28. Spammer Identification 


!/bin/bash 


by Michael S. 


Mop p opp pppHppogÀ: 
O (o 0» -1 O Qi i (). M P. O xo 0 -1 O O1 4 CQ I9 E 


Sutols Ag Spammer loans, w 1412.251:. 2004/10/01 23 94/29 93. MƏZ Ek lage: $9 
Above line is RCS info. 


The latest version of this script is available from http://www.morethan.org. 


Spammer-identification 


Zick 


Used in the ABS Guide with permission. 


HRT HE EERE EE HHH HE EEE RE EE RE EE EEE ERE EEE EE ER HER HER E A 
# Documentation 

# See also "Quickstart" at end of script. 

HGH EERE EEE HH HE EEE EE EEE HE EE EEE EE EE HH EE EEE EE HE HE HE HEE 


<<" her S panme r TBIoxer- . 
2 Copyright (c) Michael S. Zick, 2004 
22 License: Unrestricted reuse in any form, for any purpose. 
29 Warranty: None -(Its a script; the user is on their own.)- 
24 
25 Impatient? 
26 Application code: goto "f£ 4 # Hunt the Spammer' program code 4 4 #" 
2T Example output: ":««-' is spammer outputs '" 
28 How to use: Enter script name without arguments. 
29 (Que cora VOWILCKSIEsISEY dr Sro. Oi SergE 
30 


31 Provides 


de Given a domain name or IP(v4) address as input: 
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Does an exhaustive set of queries to find the associated 
network resources (short of recursing into TLDs). 
Checks the IP(v4) addresses found against Blacklist 
nameservers. 
If found to be a blacklisted IP(v4) address, 
reports the blacklist text records. 
(Usually hyper-links to the specific report.) 
Requires 
A working Internet connection. 
(Exercise: Add check and/or abort if not on-line when running script. 
Bash with arrays (2.05bt). 
The external program 'dig' -- 
a utility program provided with the 'bind' set of programs. 
Specifically, the version which is part of Bind series 9.x 
See: http://www.isc.org 
All usages of 'dig' are limited to wrapper functions, 
which may be rewritten as required. 
See: dig wrappers.bash for details. 
("Additional documentation" -- below) 
Usage 
Script requires a single argument, which may be: 
1) A domain name; 
2) An IP(v4) address; 
3) A filename, with one name or address per line. 
Script accepts an optional second argument, which may be: 
1) A Blacklist server name; 
2) A filename, with one Blacklist server name per lin 
If the second argument is not provided, the script uses 
a built-in set of (free) Blacklist servers. 
See also, the Quickstart at the end of this script (after 'exit'). 
Return Codes 


@ = VAIL OK 
i = Seriot tea luce 
2 - Something is Blacklisted 


Optional environment variables 


SPAMMER_TRACE 
iit GH tO a wrericalollSe cile; 
script will log an execution flow trace. 


SPAMMER_DATA 
It SEE CO a wenceloils iler grereijoe vili Cumo aes 
discovered data in the form of GraphViz file. 
See: http://www.research.att.com/sw/tools/graphviz 


SPAMMER_LIMIT 
Limits the depth of resource tracing. 


Default is 2 levels. 


A setting of 0 (zero) means ‘unlimited’ 
Caution: script might recurse the whole Internet! 


A limit of 1 or 2 is most useful when processing 
a file of domain names and addresses. 


PR 
= o 
OAD 


ererererrrr rrr 


[n 
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A higher limit can be useful when hunting spam gangs. 


dditional documentation 
Download the archived set of scripts 
explaining and illustrating the function contained within this script. 
http://bash.neuralshortcircuit.com/mszick clf.tar.bz2 


Study notes 


S 


A 


# 


This script uses a large number of functions. 
Nearly all general functions have their own example script. 
Each of the example scripts have tutorial level comments. 


cripting project 
Add support for IP(v6) addresses. 
IP(v6) addresses are recognized but not processed. 


dvanced project 
Add the reverse lookup detail to the discovered information. 


Report the delegation chain and abuse contacts. 


Modify the GraphViz file output to include the 
newly discovered information. 


is spammer Doc . 


PHT FE AE FE FE AE FE HHH FE FE AE FE FE AE TH HHH EH FE FE AE FE FE SE EE EE 


### Special IFS settings used for string parsing. #### 


Whitespace == :Space:Tab:Line Feed:Carriage Return: 


135. WEE IUE $e" 2207S N 9 SY N AS V AD 


136 
137, 


No Whitespace == Line Feed:Carriage Return 


138 NO WSP-S'NxOA'S'NxOD' 


139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
115) 
ISIL 
15/2 
153 
154 
15:5) 


D 


d 
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Field separator for dotted decimal IP addresses 


ADR IFS-$(NO WSP)'.' 


Array to dotted string conversions 


OT IES-'.'S(WSP TES) 

4 # Pending operations stack machine 4$ 4 # 

This set of functions described in func stack.bash. 
(See "Additional documentation" above.) 

# # 

Global stack of pending operations. 

leue f -a pending. 

Global sentinel for stack runners 
Sekera =i O (gd 


Global holder for currently executing function 


156 declare -f | pend current 


1.5/7) 
158 
15$ 
160 
161 
162 
163 
164 


# 4 Debug version only remove for regular use # # f 


The function stored in pend hook is called 
immediately befor ach pending function is 
evaluated. Stack clean, | pend current set. 


This thingy demonstrated in pend hook.bash. 


declare -f | pend hook. 
dod od 


# The do nothing function 
pend dummy () ( : ; ) 


# Clear and initialize the function stack. 
pend init() ( 

unset pending [6] 

pend func pend stop mark 

pend hook -'pend dummy' # Debug only. 


# Discard the top function on the stack. 
pend pop() { 
if [ $(£ pending IC] -gt 0 ] 
then 
logs =a EO. 
_top_=${# pending [68])-1 
unset . pending [$ top ] 
i3 


# pend func function name [$(printf '%Sq\n' arguments) ] 


pend_func() { 
local IFS=${NO_WSP} 
set -f 
_pending_[${#_pending_[@] }]=$@ 
set +f 


# The function which stops the release: 
pend_stop_mark() { 

—19- eerall 0) 
} 


pend_mark() { 
pend func pend stop mark 


) 


# Execute functions until 'pend stop mark' 
pend release() { 


lees. =i too # Declare _top_ as integer. 


_p_ctrl_=${#_pending_[@] } 
while [ ${_p_ctrl_} -gt 0 ] 
do 


.top -$(4 pending [G]j-1 
.pend current -$( pending [$ top ]! 
unset pending [$ top ] 
$ pend hook . # Debug only. 
eval $_pend_current_ 

done 


# Drop functions until 'pend_stop_mark' 
pend_drop() { 

local = CO 

local _pd_ctr1l_=${#_pending_[@] } 


while [| S{ pd ctrl} cw 0 ] 
do 
TOD- =$ joel eti -=i 
if [ "S$( pending [S$ top ])" == 'pend stop mark' 
then 
unset | pending  [$ top ] 
break 
else 


] 


2:9 unset pending [$ top ] 


23 ipxol- (e1E 1e M NOR 

233 irat 

234 done 

2535 if [ $(£ pending [80]) -eq O ] 
236 then 

237 pend_func pend_stop_mark 
238 ial 

239 jh 

240 

241 #### Array editors #### 

242 


243 # This function described in edit exact.bash. 

244 # (See "Additional documentation," above.) 

245 4 edit exact «excludes array name» «target array name» 
246 edit exact() { 


24 [ Se =ee 2 n] | 

248 | $$ -eq 3 ] || return 1 

29 local -a _ee_Excludes 

A50 local -a _ee_Target 

osi local ee x 

252 local ee t 

259 local IFS=${NO_WSP} 

254 set -f 

25/5 eval exe Ipxeiluces=\( XNSNSEN IGNI NS S 

256 val ee Target=\( \$\{$2\[@\]\} \) 

257 local |ee len-$(4 ee Target[G]) # Original length. 

258 local _ee_cnt=${#_ee_Excludes[Q@]} # Exclude list length. 
DSS) | Sf ee Jem) =e 0 |] || retum 0 # Can't edit zero length. 
260 | S4 ee en:i sane O 3 ||| zeturm 0 # Can't edit zero length. 
261 for (Up oss mx Up xc «« S4 Se eine) y Ea) 

262 do 

263 ee x-$( ee Excludes[$x]) 

264 ito (( a = 0 g sx Sf eS ew) pg aee )) 

265 do 

266 .ee t-$( ee Target [$n] } 

267 if [ x"$( ee t)" == x"$( ee x)" ] 

268 then 

269 unset ee Target [$n] # Discard match. 

DAO) [ $4 -eq 2 ] && break # If 2 arguments, then done. 
FON ie ab 

2. 2 done 

2735 done 

274 eval S2Q=\( WSWE Ge target NINE 3) 

273 set +f 

276 return 0 

xeu 

2 78 


279 # This function described in edit by glob.bash. 
280 # edit by glob «excludes array name» «target array name» 
281 edit by glob() { 


282 [ $4 -eq 2 ] |I 

283 | Sus =e, 3 ] |j return di 

284 local -a _ebg_Excludes 

285 local -a | ebg Target 

286 local | ebg x 

E local | ebg t 

288 local IFS-$(NO WSP] 

289 set -f 

290 eval | ebg Excludes-N( \$\{$1\[@\]\} \) 
291 eval _ebg_Target=\( \$\{$2\[@\]\} \) 
292 local _ebg_len=S{#_ebg_Target [@] } 
293 local _ebg_cnt=S {#_ebg_Excludes[@] } 
294 | Si ges leat sae 0 J [] recua © 
295 | S4 eese] ae U I [l| recica © 


296 toe (( & Up %& < 91 c eel g Serr )) 


(09) oy (USE (63^ (63) (09) (63) CS) 
(o9; =a) for; (Onl des Go) [S35 J 


w w 
Nor 
C «o 


do 


done 


ebg_x=$ {_ebg_Excludes [$x] } 


Tone (i 
do 


Com 


= 0 6 im SS SW xelorerdeim) 6 gue 39) 


[ S# -eq 3 ] && | ebg x-S( ebg x)'*' # Do prefix edit 


aL 
iE 


iB [ 
hen 


${_ebg_Target [$n]:=} ] i Sie Gerines & SSI. 


ebg_t=${_ebg_Target [Sn] /#${_ebg_x}/} 


[ 


iE aL 


done 


S{#_ebg_t} -eq 0 ] && unset _ebg_Target [$n] 


eval $2=\( \$\{_ebg_Target\[@\]\} \) 


Se 
Bett 


} 


+f 
rA (0) 


# This function described in unique_lines.bash. 
# unique_lines <in_name> <out_name> 


|| return 1 
in 


O_WSP } 


unique lines() 
[ $4 -eq 2 
Tocai =e, ol 
logal =a wia 
Bocal =a wil 
dRoyexdb Sak pubs 
local Cl edo 
local IFS=${N 
Bat SIE 


eval ul in-N( \$\{$1\[@\]\} V) 
wil «xxr wl lale) 


tox (( wl jos = U p ci.pos € 9i wl emt) 5 x posee )) 
do 
zi [[ $4 uL aia $4 wBü jos} i|s=} 1 # If defined & not empty 
then 
ul tmp-$( ul in[$[( ul pos)]) 
.ul out[$(£ ul out[80])]-2$(. ul tmp) 
tor (Wt wm = deos s uuo < Si cul emu) 9» wee ») 
do 
[ ${_ul_in[${zap}]:=} ] && 
[ Sw cus Simei} |} == "SUSHLOEUL emy J| Em 
unset ul in[$(zap)] 
done 
IBL 
done 
eval $2=\( \$\{_ul_out\[@\]\} V) 
set +f 
return 0 


} 


# This function described in char convert.bash. 
# to lower «string» 


to lower() { 

[ Sw ee d | [| recura di 

docegocU uk 

_tl_out=${1//A/a} 
tl out-$( tl out//B/b] 
EJL ovt= el ew € 
tl out-$( tl out//D/d] 
tl out-$( tl out//E/e) 
tl out-$( tl out//F/f) 
tl out-$( tl out//G/g) 
cil owe Si TEIL owe / mU Ia 
Dl GUESS d El GOXUTE A / 1 fa} 
el onts jell youre IA 
tl out-$( tl out//K/k) 
cil QUIET dqEJL (GOUTE/ / 1b IL} 


05 


LS 
20 
2il 
22 
IO 
24 
25 
26 
27 
28 


tl_out=${_tl_out//M/m} 
tl_out=${_tl_out//N/n} 
tl_out=${_tl_out//0/o} 
til jomuE-S tl eure / E 9t 
til quu tl gu /0//eu 
tl out-$( tl out//R/r) 
EJL xit 9 t tl out /59/s] 
El Qwip-S T El (0E /1T/tt 
tL {ae lee //u) 
tl out-$( tl out//V/v) 
tl out-$( tl out//W/w) 
tl out-$( tl out//X/x) 
Pil Oui {ie lowe 7 36/57] 
rl. guiEc ed EJ. (Utt E A 
echo $[- tl out) 

return 0 


#### Application helper functions #### 


# Not everybody uses dots as separators 
# This function described in to dot.bash 


# to dot <string> 

to wet T 
D S5 -eg 1 d || etura 1 
echo $(1//[4|8|$]/.) 


return 0 


(APNIC, for example). 


# This function described in is number.bash. 


f is number «input» 


is number() { 
p US eer R] || return 
| MSY ==  Vex(oU o qp meme aeveutiDucio. 


Local =i eg 
let tst=$1 2>/dev/null 
return $? 


iL 
0 


# is blank? 
# is zero? 


# else is numeric! 


# This function described in is_address.bash. 


# is_address <input> 
is_address() { 
1 SH eg 1 ] |) returm i 
NOGA =i . dim KNE 
local IFS=${ADR_IFS} 
_ia_input=( $1 ) 
if [ S${#_ia_input[@]} -eq 
is number ${_ia_input[ 
is number S ia input[ 
is mimber (i ia (monel 
is number $( ia input[ 
| $(—xa-cinput[0]) =1lt 25 
DI $T 3e inove =e 25 
I SX aa GuepwWE[2]] =e 25 
| 38 1 ia sepes [Si] -Lle 25 
then 
return 0 
else 
return 1 
Ta 
} 


# Blank 

] && 
) && 
) && 
) && 
} && 
6 ] && 
6 ] && 
6 ] && 
©: ] 


==> false 


# This function described in split ip.bash. 


# split ip «IP address» 


#+ «array name norm» [«array name rev»] 


eode sus 4 


[Es gH HES des qms que dí qux qEs gius duy qms Sys qne qum ies fey ghe qn dus ES Ss qi ies qs SS qus ee qe qi qf iy qim qs des qus RS diy qus ques qe duy quy qms qi quy es ges ghe. See des qi SS ie SS SES (ims qiue qx ques SS 


29 
30 


71) 


94 


| $4 -eq 3 ] [| 

I $$ eg 2°] |) ratura i 
local =i . eut alinjoule 

local IFS=${ADR_IFS} 

DSi _aligyojbte— (Sl) 
IFS=${WSP_IFS} 


# Either three 
#+ or two arguments 


eval $2=\(\ N$N(. si. inputN [GN] N) N \) 


if [ $4 -eq 3 ] 
then 


# Build query order array. 


local =e: cling). 3179) 


_dns_ip[0]=${_si_input 
elas, sto [rdi] e (Si aayoune 
Udnsiip (2A) =Ss{osi input 
Kans so 3) =9 (Sa. atoque 


eval $3=\(\ \$\{_dns_ip 


if aL 
return 0 


# This function described in 
# dot_array <array_name> 
dot_array() { 
| Su eg 1 ] |) seus i 
local — eki Nue 


[ 
[ 
[ 
[ 


x (ev [= |) 69 


NES) 


dot array.bash. 


# Single argument required. 


eval _da_input=\(\ NSN(SIN[QN]IN)N X) 


local IFS=${DOT_IFS} 


local | da output-$( da input[GQ]) 


IFS-S(WSP IFS) 
echo $ da output) 
return 0 


# This function described in 


file to array.bash 


# file to array «file name» «line array name» 


file EQ) suede (O 4d 
| SX —eg 2 1 l| ratura 1 
local IFS=${NO_WSP} 
logell cu dte JOHNS. 
Rea emon -«( Slee SI) ) 


# Two arguments required. 


eval S2=\( WS ica Emp: WEG NI NO 9 


return 0 


# Columnized print of an array of multi-field strings. 
# col print «array name» «min space» < 


#+ tab stop [tab stops]» 
Quoi jesse) 1 
| SP sgt 2 ] |) ratura Q 
local =e GS N 
local -a GJ aG 
local =a Ga lina 
local eja miko 
luoxexewdl. E Wene 
l@@aill -Ca es 
oca MECpReCNE 
IkQ@eill Ca- ce 
Foca al — je) 
logal =i. GE 
osul eja ikel 
# WARNING: FOLLOWING LIN 


FANOTFETANKE arab IES) (OXU(OXL 


ilo cast GO Ina a 

set -f 

local IFS-$(NO WSP] 

eval _cp_inp=\(\ \$\{$1\ 
[ ${#_cp_inp[@]} -gt O ] 


TENSES Y 


HD) SevGis 


|| return 0 # Empty is easy. 


495 . cp mcent-$2 

496 (19 umane 94. Clo mexi gS {cys memei} 

497 ‘Sines 

498 shift 

499 _cp_cnt=S# 

S00 iw (ME ceo = OW c Go < e (em. D epu ) 

501 do 

502 Tepr Se (Sti cjo_ Soe! LE! T] JT] e Stes meee Ze Si cogi 
SOS shift 

504 done 

505 Eicpmenis s Ere oer [| } 

B UNS ror (( des = OW pf epik Lucie Cite P epus ) 

507 do 

508 _cp_pos=1 

SOS: IFS=$ {NO_WSP}$'\x20' 

510 ep Jisumec( p ep amps col) J 

51. 1t IFS=$ {NO_WSP } 

512 ioe (( xeu = © pr epi < 90 e e E 2 ena )) 
Sl 3 do 

514 (jo; icalo=S | _clo_sine [LS 4. reyexe } 1] 9/9. «er 19oxoxs JO 
515 xi | Siu cp ie ike S14 c we ] 

516 then 

ll 7/ CO Eso US «ep mat 

Se if aL 

519 exelmg. sia "S cp Teu 

520 (( eo- pos = Si oa POSi + cp iai) 
DZI -ep Cicus co linssi eor] p» 

522 Scho =i Si cio 3l) 

526 (( Ginx isos = Snep posh s Siu ce-sclel- ») 
524 done 

525 echo 

526 done 

52:1) set +f 

520 return 0 

52:98 

530 

Ss 4 4 # 'Hunt the Spammer' data flow 4 # 4 # 

522 

533 Application return code 

HS! (elexedheuess Sal las INC 

535) 

536 Original input, from which IP addresses are removed 
ug After which, domain names to check 

538 declare -a uc, name 

539 

540 Original input IP addresses are moved her 

541 After which, IP addresses to check 

542 declare -a uc address 

543 

544 Names against which address expansion run 

545 Ready for name detail lookup 

546 declare -a chk name 

547 

548 Addresses against which name expansion run 

549 Ready for address detail lookup 

550 declare -a chk address 

55A 

552 Recursion is depth-first-by-nam 

553 The expand_input_address maintains this list 
554 #+ to prohibit looking up addresses twice during 
555 #+ domain name recursion. 

556 declar a been_there_addr 

557 been there addr-( '127.0.0.1' ) # Whitelist localhost 
556 


559 # Names which we have checked (or given up on) 
560 declare -a known_name 


Addresses which we have checked (or given up on) 
declare -a known address 


List of zero or more Blacklist servers to check. 

Each 'known address' will be checked against each server, 
+ with negative replies and failures suppressed. 

declare -a list server 


directi onmi set to zero == no limit 
indirect-$(SPAMMER LIMIT:-2)] 


4 4$ # 'Hunt the Spammer' information output data # 4 # 4 


Any domain name may have multiple IP addresses. 
Any IP address may have multiple domain names. 
Therefore, track unique address-name pairs. 
declare -a known pair 

declar a reverse pair 


In addition to the data flow variables; known address 
* known name and list server, the following are output to the 
+ external graphics interface file. 


Authority chain, parent -» SOA fields. 
declare -a auth, chain 


Reference chain, parent name -> child name 
declare -a ref chain 

DNS chain - domain name -> address 
declar a name address 

Name and service pairs - domain name -> service 
declar a name srvc 

Name and resource pairs - domain nam > Resource Record 
declar a name resourc 

Parent and Child pairs parent nam » child name 

This MAY NOT be the same as the ref chain followed! 
declar a parent child 


Address and Blacklist hit pairs address-»server 
declar a address hits 


Dump interface file data 
declare -f | dot dump 
.dot dump-pend dummy # Initially a no-op 


# Data dump is enabled by setting the environment variable SPAMMER DATA 
#+ to the name of a writable file. 
declare dot file 


# Helper function for the dump-to-dot-file function 
# dump to dot «array name» «prefix» 
lone de ek) 4 

local -a | dda tmp 

local -i -dda cnt 

ikoeavik olke it@renn=! 'S(2)'$04u %Ss\n' 

local IFS=$ {NO_WSP } 

eval _dda_tmp=\(\ \S\{S1\[@\]\}\ V) 

_dda_cnt=$ {#_dda_tmp[@] } 

ai [ S4 la eine} sie 0 ] 

then 

rom (E eee = © 5. lela, «€.- (eleiei- deus p. . lola )) 


627 do 

628 joie US ele sco} \ 

629 WIS gels Wes lets emo (|S d elei IP" ew elo sen Ike) 
630 done 

Gaul i3 

632 } 

633 

634 4 Which will also set | dot dump to this function 

635 dump dot() ( 


636 logal =a CEL cine 

637 echo '# Data vintage: 'S(date -R) >S{_dot_file} 

638 echo '# ABS Guide: is_spammer.bash; v2, 2004-msz' >>${_dot_file} 
639 echo »»5$( dot file) 

640 echo 'digraph G (' »»5$( dot file) 

641 

642 if [ ${#known_name[@]} -gt 0 ] 

643 then 

644 echo »»5$( dot file) 

645 echo '# Known domain name nodes' »»$( dot file) 

646 _dd_cnt=${#known_name[@] } 

647 for Dp gg eel; @lel «€ lel (exe 3 ddt) 

648 do 

649 primet V N$04u [label-"$s"] ;\n' \ 

650 "S( dd)" "S${known_name[${_dd}]}" »»5$( dot file) 
651 done 

652 fi 

653 

654 if [ ${#known_address[@]} -gt O ] 

655 then 

656 echo »»5$( dot file) 

657 echo '# Known address nodes' »»$( dot file) 

658 _dd_cnt=${#known_address[@] } 

659 roe (0 eels © p _elel < . iol eae p oE )) 

660 do 

661 jexcinonpie v aO [lalssl="Sa"] sm" X 

662 "$( dd)" "S{known_address[${_dd}]}" »»5$( dot file) 
663 done 

664 iE 3l. 

665 

666 echo 20591 eon rabie 

667 exem. Vf »»S$1 glo sea. lS}; 

668 echo ' * Known relationships :: User conversion to' »»$( dot file] 
669 echo ' * graphic form by hand or program required.'  »»$( dot file) 
670 acino wl »»5( dot file) 

671 

672 aie [p trauca chaim xe 0 | 

673 then 

674 echo »»5$( dot file) 

$75 echo '# Authority ref. edges followed & field source.' »»5$( dot file) 
676 dump to dot auth chain AC 

677 deat 

678 

679 zd | Swgxes emssua[e] er O ] 

680 then 

681 echo »»5$( dot file) 

682 echo '# Name ref. edges followed and field source.' »»$( dot file) 
683 dump to dot ref chain RC 

684 fi 

685 

686 if [ ${#name_address[@]} -gt 0 ] 

687 then 

688 echo >>${_dot_file} 

689 echo '# Known name->address edges' >>${_dot_file} 

690 dump_to_dot name_address NA 

691 fi 


692 


dges' »»5$( dot file) 


xs dp Sijadeune sieve (Cl) )} seis © 1 
then 
echo »»$( dot file) 
echo '# Known name->servic 
dump to dot name srvc NS 
ie 
if [ ${#name_resource[@]} -gt 0 ] 
then 
echo »»5$( dot file) 
echo '# Known name->resourc 
dump to dot name resource NR 
if. 


if [ ${#parent_child[@] } 
then 
echo »»5$( dot file) 


one: 0 J 


dges' >>${_dot_file} 


echo '# Known parent-»child edges' >>${_dot_file} 
dump to dot parent child PC 


if 


if [ ${#list_server[@]} -gt 0 ] 


then 
echo >>${_dot_file} 


echo '# Known Blacklist nodes' 


eel cac=St list server 0} 


Tow ( e = 9 p ee « 


(olgl. «how 


ESS | «ele ac LS; 


& elem 3) 


do 


Drine | LS%04u 


[lalxel= Se") sm" Y 


"S{_dd}" "S{list_server[${_dd}]}" >>S${_dot_file} 


done 
if aL 


unique_lines address_hits address_hits 


if [ ${#address_hits[@] } 
then 
echo >>${_dot_file} 


seit 0 ] 


echo '# Known address->Blacklist_hit edges' >>${_dot_file} 
echo '# CAUTION: dig warnings can trigger false hits.' >>${_dot_file} 
dump to dot address hits AH 


iE aL 

echo 25894 he, eile) 

exeo. V cU 2o xe me 

acho " = "neue as a lor (ous relationshtos. diejerx Gradara 2294 o, ene] 
icine "U A SSS dot ale] 

echo 'j' Sms eu dade 


return 0 


# 4 # # 'Hunt the Spammer' execution flow # 4 # # 


# Execution trace is enabled by setting the 


#+ environment variable SPAMM 
declare -a trace log 
declare | log file 


ER TRAC 


# Function to fill the trace log 


trace logger() { 


to the name of a writable file. 


.trace log[$(f£ trace log[G])]-$(. pend current ) 


# Dump trace log to file function variable. 


declare -f | log dump 


# Dump the trace log to a file. 


log dump-pend dummy 4 Initially a no-op. 


759 cmo log 1 


760 
761 
762 
763 
764 
765 
766 
767 
768 
769 
770 
Ji Wal 
YZ 
TAS 
774 
WS) 
TVG F 
7733] 
WS 
TIO 
780 
TASIE 
782 
783 
784 
785 
786 
787 
788 
m9 
790 
3/93. 
192 
793 
794 
795 $ 
796 
VST) 
798 
799 
800 e 
801 # 
802 
803 
804 
805 
806 
807 # 
808 
809 
10 


oo 


OO OO © © © OO OO CO 
(cei SJ) (ox (ub des Go IS) [E 


819 
920) jj 
OD 
822 # 


Toca al, _telil_ (eite 
.dl cnt-$(£ trace log[G]) 


FOC SEO mde erus cns ciere CENE D 


do 


echo $( trace log[$( dl1)]) >> $( log file] 


done 
.dl cnt-$(£ pending [G8]) 
ie [ Siel ence age © 1 
then 

(oL exa S d oll seine} =a 


echo '# # 4 Operations stack not 


irene (¢ li = Sf LL cuj 
do 


echo ${_pending_[${_ 


done 
fai 


p. ei SS 


oui » 


empty 4 # #' 
obl y) 


0 ; 


$( log file) 


4 # Utility program 'dig' wrappers 4 # 4 


Ihe major difference is thes 


These wrappers are derived from the 
+ examples shown in dig wrappers.bash. 


return 


# # 


3e eierne TSSuulhes evs El Lee- alia iar eucmeeny. 


See dig_wrappers.bash for details and 
+ use that script to develop any changes. 


Short form answer: 'dig' parses answer. 


Forward lookup 


Name -» Address 


short fwd «domain name» «array name» 


tert wd 
local -a sf reply 
loca IMI SE ele 
local “ak, Sf ent 
IFS=${NO_WSP} 

(imo; =i UU 

echo 'sfwd: '${1} 

_sf_reply=( $(dig *short ${1} 

EST sees? 

ie |) eMe dece Sane ] 

then 


_trace_log[${#_trace_log[@]}]='## Lookup error 


>> $( log file) 


-c in -t a 2»/dev/null) 


| Sí suae] me 9 1 && jewel Croo 


TECUM (S eue see}! 

else 
# Some versions of 'dig' 
-ei gquE-STq s repliye) 


return warnings on stdout. 


ione ((( We = 0 2 se < 94 E cine} 
do 
| as wee ced ed su s0821 
unset sf reply[$( sf]] 


done 


, 


eval $2=\( \$\{_sf_reply\[@\]\} V) 


if, 
return 0 


Reverse lookup 


Address -» Name 


823 # short rev «ip address» «array name» 


824 s 


hort rev() { 


SIE 


eS SMS CoA ey 


)) 


) 


on 


JSt ge 


#E' 


825 
826 
B27) 
828 
829 
830 
831 
832 
833 
834 
835 
836 
837 
838 
839 
840 
841 
842 
843 
844 
845 
846 
847 
848 
849 
850 
851 
852 
853 
854 
855 
856 
857 
858 
859 
860 
861 
862 
863 
864 
865 
866 
867 
868 
869 
870 
ewa 
SUZ 
873 
874 
875 
876 
877 
878 
979) 
880 
881 
882 
883 
884 
885 
886 
887 
888 
889 
890 


local -a sr reply 
Tocak Sal Hr 3e 
loca =a, cepe CNE 
IFS-S$(NO WSP) 

Sele. ie UU 

# echo 'srev: 


"S{1} 


_sr_reply=( $(dig +short -x ${1} 2>/dev/null) ) 


ES r cies 


zi [ 94 sri xe] -—ee 0] 
then 
.trace log[$(f£ trace log[G])]-2'44 Lookup error '$( sr rocki on '$(1)' ##' 
# [ $0, sr. rc) ne 9 ] && pend drop 
eeu S Jede. ere 
else 
# Some versions of 'dig' return warnings on stdout. 
_sr_cnt=${#_sr_reply[@] } 
iue (Mb ens. = 0) RO sae <i ee iene} W- denen. )) 
do 
| Veit sue secerenbw bed eed SWS 2) SS Yes qose 
unset _sr_reply[${_sr}] 
done 
gyal SW MP er reply EVI NE X) 
ita 
return 0 
} 
# Special format lookup used to query blacklist servers. 
# short text «ip address» «array name» 
short text(» f 
local -a | st reply 
dhoyemul ak, sie, see! 
toca =a Sit eine 
IFS=$ {NO_WSP} 
s oho "ees USTqIL 
etr wejolly=( Schie s ehore SAk -e im Sie E 27/0 swLI) ) 
_St_rc=5? 
Sse [Sal ie edere cete (0! 
then 
_trace_log[${#_trace_log[@]}]='##Text lookup error '$( st rc)' on ES 


# [ $( st rc) -ne 9 ] && pend drop 
Teeven S ge ice} 

else 
# Some versions of 'dig' 


_st_cnt=${#_st_reply[@] } 


f oT ME REESE OC eh EC eS pu CTI 
do 
| "ss" Si mue seempusviet st!]s0s2] 
unset st reply[$( st)] 
done 


eval $2=\( \$\{_st_reply\[@\]\} V) 
ip ak 
return 0 


Tox LONE OFAN, Balen @az 


RUC 2 S Service lookups 
«service». «protocol».«domain name» 
.ldap. tcp.openldap.org. 3600 IN 


Forward lookup 
long fwd «domain name» «array name» 
long fwd() { 

docs =a . 4E seed 


, 


SRV 
domain TTL Class SRV Priority Weight Port Target 


return warnings on stdout. 


Sitar ))) 


the parse it yourself versions 


dig +tnoall +nofail tanswer  ldap. tcp.openldap.org -t srv 


0 0 389 ldap.openldap.org. 


Name -> poor man's zone transfer 


Lecet Sal _ lit ee 
Toca Mci IE ent 
IFS=${NO_WSP} 
echon =n Nei 
s eeno "Juwel "Sq 
Alt szweply( Sq 


dig +noall +nofail *answer tauthority +tadditional \ 
S$(1) -č soa ${1} -t mx S{1} -t any 2>/dev/null) ) 


lf rc-$? 
ad dp 94 dE we} =e © ] 
then 


_trace_log[${#_trace_log[@]}]='# Zone lookup err 


i? [ SW dE aeg me 9 ] ee gue cheers 
ertza (S LE rel 
else 


CE due ego 


# Some versions of 'dig' return warnings on stdout. 


Dr ent Sd are pna] 
EOC ((( EESO ile eee Sh edle eine! vit emt dE RS 
do 
enS rep Eye a ibe} 0e 23. Ex Viste 
passt . JE seed [e 358 
done 
eval, $2=\ 0 WSVXI db] seegeudby NE GNI \i 3) 
1E 3L 
return 0 


The reverse lookup domain name corresponding to the IPv6 address: 


} 
# 
# 4321505385228 sie 4o 567 9999 
# would be (nibble, I.E: Hexdigit) reversed: 
# 


)) 


on 


"Ep 


aa 


loto € o o Ios 9.0. 2.10) 0). (0). 5) 50] 5 001,5 (0).5 27 510) 5 (0). (0). 1L) 0.0.0.0 0.051.263 dE o JL AREA, 


# Reverse lookup :: Address -> poor man's delegation chain 


# long rev «rev ip address» «array name» 
long rev() { 
koca =e, ie sespouby 
koce Sak di ee 
ocal ak ilse Tenue 
ipval. ele cos 
le emess 1L JEU atinieitclolie,, eisejorel . ! 
IFS=S {NO_WSP } 
Echon ane "gt 
s Goo "Jews Si il} 
-lr ceply=( $ 


dig +noall +nofail +tanswer tauthority -*additional \ 
Stile cms) -~t s@e Si lr clas} =e any 2/clew/oilil) 3} 


_lr_rc=$? 
ie || S4 2338 we} m © ] 
then 


_trace_log[${#_trace_log[@]}]='# Deleg lkp error 


# [ SC. 1r rc) -ne 9 ] && pend drop 
cerura de die seg 
else 


HS jie xg] 


# Some versions of 'dig' return warnings on stdout. 


.lr cnt-$(£ lr reply al) 
for (( he = o0 6 ie € S42 ise emt}; 9 diese 
do 
[ES ONES e posi [ES a geal) E Asters 
wu le seply[eq lx] 
done 
eval $2=\( \$\{_lr_reply\[@\]\} V) 
iE aL 
return 0 


# # # Application specific functions # # # 


)) 


on 


SEAL IO 


qa ll 


957 4 Mung a possible name; suppresses root and TLDs. 
958 # name fixup <string> 
959 name fixup()( 


960 
961 
962 
963 
964 
965 
966 
967 
968 
969 
DTO 
9 qi 
912 
973 
974 
OFS) 
976 
SUT 
978 
99 
980 
981 
982 
Diss 
984 
985 
986 
987 
988 
28$ 
990 
ION 
292 
999 
994 
995 
996 
SY) 
999 
999 


local =a odeur Menya 
local -i —nf-end 
ioco rias ES hT 
loeeul ines) 
nf str-$(to lower ${1}) 
dimit Str=S (ue cot S xmi eel) 
debt emos ue go qenpae d 
| Sd sme erres ue ed l=] VU. ee 
iut eric sau STE) "sU 
IFS=${ADR_IFS 
ioe eas Sd sour genere) 
IFS-S$(WSP IFS 
_nf_end=${#_nf_tmp[@] 
case $( nf end) in 
O s» NO oko «exo (los 
echo 
return 1 


fo 


) ai? Oily «v IND). 
echo 
return 1 


N 


) # Maybe okay. 
exo Gi an gu] 
return 0 
# Needs a lookup table? 
if | $($-nf tmpli]) -eq 2 | 
then # Country coded TLD. 
echo 
return 1 
else 
eero (f. sae eea 
return 0 
fe 
E 
esac 
echo $( nf str) 
return 0 


1000 4 Grope and mung original input (s). 
LOO eyed mawe) 1 


1002 
1003 
1004 
1005 
1006 
1007 
1008 
1009 
1010 
WOLT 
1012 
1013 
1014 
1015 
1016 
1017 
1018 
1019 
1020 
1021 
1022 


[ Sis meme (Cl; -er © | || return © 
dioxemwdl 3L. ab eom 

tocak =i si lemn 

lioxeeudL . 3L Siew 

unique lines uc name uc name 
_si_cnt=S${#uc_name[@] } 

aoe (( 8h S © 2 Sit < usa vex ¢ 4s )))) 
do 


_si_str=${uc_name[$_si] } 

if is_address S{_si_str} 

then 
uc_address [${#uc_address[@]}]=${_si_str} 
unset uc_name[$_si] 


else 
ii | ve omevake [Let ] 59 (meme Timo S1 Si stri) 
then 
unset ucname[$ si] 
ik 
iE 
done 


uc name-( S{uc_name[@]} ) 


_si_cnt=S${#uc_name[@] } 


_trace_log[${#_trace_log[@]}]='#Input 'S{_si_cnt}!' 


_si_cnt=${#uc_address[@] } 


_trace_log[${#_trace_log[@]}]='#Input 'S{_si_cnt}!' 


return 0 


unchkd name input(s 


unchkd addr input(s 


Recursion limiter 

limit chk() «next level» 
Tome elo (0) d 

theese =a, _ Ike dB 

# Check indirection limit. 


# # Discovery functions recursively interlocked by external data 
# # The leading 'if list is empty; return 0' in each is required. # # 


ie || Spese] eer 9 3 || | Si es 9 | 
then 
# The 'do-forever' choice 
echo 1 # Any value will do. 
return 0 # OK to continue. 
else 
# Limiting is in effect. 
ie || S43ugebbxeerj lie Stil} ] 
then 
echo $(1) # Whatever. 
return 1 # Stop here. 
else 
.lc lmt-$(1)-41 # Bump the given limit. 
scho S4 le Ime s NSIS) 31 
return 0 # OK to continue. 
fi 


iE aL 


For each name in uc_name: 

Move name to chk_name. 

Add addresses to uc_address. 

Pend expand_input_address. 

Repeat until nothing new found. 
expand_input_name <indirection_limit> 
expand_input_name() { 

[ ${#uc_name[@]} -gt 0 ] || return 0 
local -a _ein_addr 

local -a | ein new 

local —i “wen cnt 

ilocal =2, Guo CNE 

local ein tst 

_ucn_cnt=$ {#uc_name[@] } 


adr — d 1930. (RES (sumam lake S) 
then 
return O0 
i3 
tor (WX ein = © s eina uen cine p -aime y) 
do 
if short, fwd ${uc_name[${_ein}]} . ein new 
then 
for (( ein cnt = 0 ; ein cnt « ${#_ein_new[@]}; | ein cnt-4- 
do 
ium ibe-S$ eue newel guum Cnty 
abs ale. eyelchaesis: Gub oln CsE) 
then 
_ein_addr[${#_ein_addr[@]}]=${_ein_tst} 
fi 
done 


# 


) 


Ic 


# 
# 


)) 


gall 


gel 


# 


1089 iur 


1090 done 

1091 unique lines ein addr | ein addr # Scrub duplicates. 

1092 edit exact chk address | ein addr # Scrub pending detail. 
1093 edit exact known address | ein addr # Scrub already detailed. 
1094 if [ Siy ein addr[8]) -gt O ] # Anything new? 

1095 then 

1096 uc address-( ${uc_address[@]} $í( ein addr[Q0]) ) 

1097 pend func expand input address $(1) 

1098 _trace_log[${#_trace_log[@]}]='#Add '${#_ein_addr[@]}"' unchkd addr inp.#!' 
1099 ral 

1100 edit_exact chk_name uc_name # Scrub pending detail. 
AREON edit_exact known_name uc_name # Scrub already detailed. 
i102 if [ ${#uc_name[@]} -gt 0 ] 

1103 then 

1104 chk name-( ${chk_name[@]} ${uc_name[@]} ) 

NOS pend func detail each name $(1) 

1106 fase 

INOJ unset uc, name[Q] 

1108 return 0 

1109 

ELO 

een For each address in uc address: 

I DD Move address to chk address. 

IUIS Add names to uc name. 

4 Pend expand_input_name. 

ILIES Repeat until nothing new found. 

LLLE expand_input_address <indirection_limit> 

1117 expand_input_address() { 

ELG [ S${#uc_address[@]} -gt 0 ] || return 0 

LILLY) local -a _eia_addr 

LAL 20) local -a _eia_name 

NIZI local -a , eia new 

13272 local =i eve. (usur 

1.1123; Focal <I eia Cnt 

124 toce Ote Cer 

i125 unique_lines uc_address _eia_addr 

1126 unset uc, address[Q] 

127 edit exact been there addr | eia addr 

L2G _uca_cnt=${#_eia_addr[@] } 

1129 | Si wea cme} =c 0 p se 

LIL SO) been there addr-( $(been there addr[G]) S{_eia_addr[@]} ) 
aL Sal 

ILSA Rea (( wm = 0 ¢ cua < Wea Cte p exe )) 

LASS do 

1.1.84 if short rev $( eia addr[$( eia)]) .eia new 

111.95 then 

1136 oie (( eile cine = 0 5 eB ene S «€ S1 Sue sos ie} pn rere )) 
JL1 3/7) do 

1138 ia tst-$( eia new[$( eia cnt)]) 

1139 hit eia Ttst-9 (mene fixup O1 aila coti) 

1140 then 

Lakai eila namals t sia nena [est eta test? 

ILAZ E 

1143 done 

1144 3E 3L 

1145 done 

1146 unique lines eia name eia name # Scrub duplicates. 

1147 edit exact chk name | eia, name # Scrub pending detail. 
ILAG edit exact known name eia name # Scrub already detailed. 
1149 if [ ${# eia_name[@]} -gt O ] 4 Anything new? 

1150 then 

Li Sal uc name-( S${uc_name[@]} ${_eia_name[@]} ) 

13,52 pend func expand input name ${1} 

1353 _trace_log[${#_trace_log[@]}]='#Add '${#_eia_name[@]}' unchkd name inp.#' 
1154 ít 3L 


1.55 
4.55 
157 
158 
1.59) 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170) 
Eyi 
LA 
LYS 
174 
LYS 
176 
AL q) 
178 
JL 7/8) 
180 
181 
182 
183 
184 
185 
186 
187 
188 
189 
1.9) 
JL 
192 
19.3 
194 
1.95 
1.95 
1.99 
1.98) 
199) 
1200 
LZ OAL 
1202 
1203 
1204 
1205 
1206 
1207 
1208 
1209 
1210 
NZILA 
162162 
ALS 
1214 
12S 
1216 
11271. 3 
1218 
123.6) 
1220 


ld p HB p HB p p HB np p|B p pB pH p dH p dB p P H pP BH H pH pP PB H pP pH pH pPB pP PH BH | pH H pm|B RPP PEPE 


m 


edit_exact chk_address _eia_addr # Scrub pending detail. 


edit_exact known_address _eia_addr # Scrub already detailed. 
aie [| Sip eia acbkh iG JE ene. © | # Anything new? 
then 


chk address-( S{chk_address[@]} ${_eia_addr[@]} ) 
pend func detail each address ${1} 

ít 

return O0 


# The parse-it-yourself zone reply. 


# The input is the chk name list. 
# detail, each name «indirection limit» 
detail each name() { 
S{#chk_name[@]} -gt 0 || return 0 
local -a _den_chk Names to check 
local -a _den_name Names found here 
local -a _den_address Addresses found here 
local -a den pair Pairs found here 
local -a . den rev Reverse pairs found here 
local =a . «eem. im Line being parsed 
local -a , den auth SOA contact being parsed 
local -a , den new The zone reply 
local =e esu joe Parent-Child gets big fast 
local -a _den_ref So does reference chain 
local -a | den nr Name-Resource can be big 
local -a | den na Name-Address 
local =e . eum dm Name-Service 
local «— cem Clue Chain of Authority 
loeal =i _clein cic Count of names to detail 
local =i -Cam dg Indirection limit 
local _den_who Named being processed 
local _den_rec Record type being processed 
local _den_cont Contact domain 
koce Clen EXE Fixed up name string 
local _den_str2 Fixed up reverse 
local IFS-S(WSP IFS) 
Local, unique copy of names to check 
unique lines chk name . den chk 
unset chk name[Q] # Done with globals. 
# Less any names already known 
edit exact known name | den chk 
.den cnt-$(£ den chk[G8]) 
4 If anything left, add to known name. 
[ ${_den_ent} -gt 0 ] && 
known name-( ${known_name[@]} $( den chk[G]) ) 
# for the list of (previously) unknown names 
TOI p CHIC s den « den cnt ; dent++ )) 
do 
den who-$( den chk[$( den])]) 
if long fwd $( den who) | den new 
then 
unique lines den new | den new 
if [ $(£ den new[80]) -eq O ] 
then 
_den_pair[${#_den_pair[@]}]='0.0.0.0 '$(. den who) 
ít 
# Parse each line in the reply. 
for (( line = 0; ine < ${#_den_new[@]} ; _line++ )) 
do 
IFS-$ {NO_WSP}$'\x09'$'\x20!' 


.den tmp-( ${_den_new[${_line}]} ) 


IFS-S(WSP IFS 


) 


4 If usable record and not a warning message 
if [| $($-den tmp[0]) -gt 4 ] && | 


then 


den rec-$í( den tmp[3]) 


'x'$( den tmp[0]) != 


den nr[$(4 den nr[80])]-2$(. den who)' '$(. den rec) 


# Begin at RFC1033 (+++) 
ease SI dem rec im 


#<name> [«ttl»] [<class>] SOA «origin» <person> 
SOA) # Start Of Authority 
if _den_str=$ (name_fixup ${_den_tmp [0] }) 


then 


_den_name[${#_den_name[@] }]=${_den_str} 


_den_achn[${#_den_achn[@]}]=${_den_who}' 
# SOA origin -- domain name of master zone record 
if den str2-$ (name fixup ${_den_tmp[4]}) 


then 


. den name[$(£f den nam 


den achn[$(£ den achn[ 


feat 


@ 
@ 


] 
] 


]2$( den str2 


} 
}]=S{_den_who} 


# Responsible party e-mail address (possi 
# Possibility of first.last@domain.name ignored. 


gie SIE 


if den str2-$ (name fixup $( den tmp[5])) 


then 
IFS=${ADR_IFS} 


_den_auth=( ${_den_str2} 


IFS-S(WSP IFS) 
ie dp S45 lm uum e } 
then 


) 


-GeE 2 ] 


den, cont-$( den auth[1]) 
nor (0 awien = 2^ p awen < Sie? ker cuum] s uem )) 


do 


7$ hem erri” SOW! 


) 


" $1 cen med] SOA O. 


lolly? lexexenutsi)) c 


den, cont-$( den cont)'.'$( den auth[$(. auth]]) 


done 


. den name[S(£ den name[GQ 
_den_achn[${#_den_achn[@] 


ie 
set +f 
fi 


A) 4$ IP(v4) Address Record 
if den str-$(name fixup $ 


then 


[@]}]=${_den_co 


} 
}]=${_den_wh 


irat 


_den_tmp[0]}) 


_den_name[${#_den_nam 
_den_pair[${#_den_pai 

den_na[${#_den_na[@ 
_den_ref[${#_den_ref 


ES 


} 
@ 


@ 
@ 
} 


}]=S{_den_str} 
j-] 95 cen Cmo l 
St (le suc" "US 
-$( den who]' 


else 
.den pair[S(4 den pai 
den na[S$(4 den na[G 
den ref[$[(f den ref 


T 


} 
@ 


@ 
} 


j = (clei cmol 
'unknown.domain 
-$( den who]' 


iE 


ile Pe 


9" Ve ks gum". SONS 


av den rc) 
{_den_tmp [4] } 
"Sf Clem gui" JAY 


4])' unknown.domain' 
^S eleva, desir 4 || 3 
unknown.domain A' 


_den_address[${#_den_address[@]}]=${_den_tmp [4] } 


den_pc[${#_den_pc[@] } 


vr 


-$( den_who}' 


NS) # Name Server Record 


# Domain name being serviced (may 
if _den_str=$ (name_fixup ${_den_tmp[0]}) 


then 


EIE 


den tmp[4]) 


be other than current) 


den name[$(£ den name[80])]-$(. den str) 
_den_ref [${#_den_ref [@]}]=${_den_who}' '$( den str)' NS' 


# Domain name of service provider 
if den str2-$ (name fixup s den tmp[4])) 
then 
_den_name[${#_den_name[@]}]=${_den_str2} 
em rer (Sih com eet [GI] sea wine!" "S4 ciem ser2" Wes 


@ 
den_ns[${#_den_ns[@]}]=${_den_str2}"' NS' 
den_pc[${#_den_pc[@]}]=${_den_str}' '${_den_str2} 

ical 
fa 


vr 


MX) # Mail Server Record 
# Domain name being serviced (wildcards not handled here) 
if _den_str=$ (name_fixup ${_den_tmp[0]}) 
then 
_den_name[${#_den_name[@] }]=${_den_str} 
_den_ref[${#_den_ref[@]}]=${_den_who}' '$( den str)' MX' 
1E dL 
# Domain name of service provider 
if | den str-$ (name fixup $( den tmp[5])) 
then 

den name[$[(4 den name[Q])]-2$(. den str) 

Cem cerlo so dem rera =e oen valer) VS den srr) Midt 
dem mels ii cern aee (El) 199 cen ess) U MY 
den_pc[${#_den_pc[@]}]=${_den_who}' '$(. den str) 

1E 3L 


vr 


PTR) # Reverse address record 
# Special name 
Lit Clini dense (acme eisai Si Cen iw] 1) 
then 
Oea rarei Cen rer LT] JH] SW den Wino}! “Si demn stih" IIT! 
# Host name (not a CNAME) 
if _den_str2=S (name_fixup $( den tmp[4])) 
then 
Cen aee | SG: Cen sees [el =s cen sus] "US lem sz) 
.den ref[$(4 den ref[G])]-$( den who)' '$( den str2)' PTRH' 
dema joe [LS Cs cea pelti l=s4_clem_ wno "SUE cen srr? 


feat 


Fr 


AAAA) 4$ IP(v6) Address Record 
if den str-$(name fixup $( den tmp[0])) 
then 
den name[$(4 den name[Q])]-2$(. den str) 
_den_pair[${#_den_pair[@]}]=${_den_tmp[4]}' '$( den str) 
} 


den_na[${#_den_na[@]}]=${_den_str}' '$( den tmp[4]) 
demn refle r den sese [p => Cen wo" Vsi dam eue" uuu 


else 
_den_pair[${#_den_pair[@]}]=${_den_tmp[4]}' unknown.domain' 
den na[$(ft den na[G])]-'unknown.domain '${_den_tmp[4] } 


den ref[$(4 den ref[8])]-$(. den who)' unknown.domain' 
3E 3L 
# No processing for IPv6 addresses 

den pc[$(4 den pc[G])]-$( den who)' '${_den_tmp [4] } 


mr 


CNAME) # Alias name record 

# Nickname 
if _den_str=$ (name_fixup ${_den_tmp[0]}) 
then 


den name[$(£ den name[80])]-$(. den str) 


CNAME' 


(HOS 


_den_ref [${#_den_ref [@]}]=${_den_who}' '$( den str)' 
den pc[$(4 den pc[8])]-$(. den who)' TSi den str) 
i3 
# Hostname 
if _den_str=$ (name_fixup ${_den_tmp[4]}) 
then 
_den_name[${#_den_name[@] }]=${_den_str} 
_den_ref [${#_den_ref [@]}]=${_den_who}' '$( den str)' 
den_pc[${#_den_pc[@] }]=${_den_who}' '${_den_str} 
i3 
TXT) 
esac 
Ea 
done 
else # Lookup error == 'A' record 'unknown address' 
_den_pair[${#_den_pair[@]}]='0.0.0.0 '$(, den who) 
fi 
done 
# Control dot array growth. 
unique lines | den achn , den achn # Works best, all the same. 
edit exact auth chain den achn 4 Works best, unique items. 


if [ $(£ den achn[8]) -gt 0 ] 

then 
IFS=S {NO_WSP } 
auth_chain=( S${auth_chain[@]} ${_den_achn[@]} ) 
IFS=${WSP_IFS} 


i aL 
unique_lines _den_ref _den_ref # Works best, all the same. 
edit_exact ref_chain _den_ref # Works best, unique items. 


alae [ Sse cea sees [el e 0 | 

then 
IFS=S {NO_WSP } 
rer Cligiia—( lrer cba] $4 cen sees [Gl ) 
IFS-S(WSP IFS) 

ita 


unique lines | den na , den na 

edit exact name address | den na 

if [ $(£ den na[8]) -gt 0 ] 

then 
IFS=S {NO_WSP } 
name address-( ${name_address[@]} ${_den_na[@]} ) 
IFS=${WSP_IFS} 

i3 


unique lines den ns | den ns 

edit exact name srvc den ns 

xx | Sis eem cse } <w © | 

then 
IFS-$(NO WSP) 
name srvc-( ${name_srvc[@]} $[(. den ns[Q]) ) 
IFS=$ {WSP_IFS} 

apie 


unique_lines _den_nr _den_nr 


edit exact name resource _den_nr 
ie [D Sb cen mele -e O | 
then 
IFS-$(NO WSP) 
name resource-( ${name_resource[@]} ${_den_nr[@] } 


IFS-S(WSP IFS) 


) 


1419 
1420 
1421 
1422 
1423 
1424 
1425 
1426 
1427 
1428 
1429 
1430 
1431 
1432 
1433 
1434 
1435 
1436 
1437 
1438 
1439 
1440 
144 
144 
144 
144 
144 
144 
144 
144 
1449 
1450 
1451 
1452 
1453 
1454 
1455 
1456 
1457 
1458 
1459 
1460 
1461 
1462 
1463 
1464 
1465 
1466 
1467 
1468 
1469 
1470 
1471 
1472 
1473 
1474 
1475 
1476 
1477 
1478 
1479 
1480 
1481 
1482 
1483 
1484 


oOo —D oy, OS WN ES 


EI 


unique lines | den pc | den pc 

edit exact parent child den pc 

if [ S$S(£ den pc[G]) cw 0 ] 

then 
IFS-$(NO WSP) 
parent child-( ${parent_child[@]} ${_den_pc[@]} ) 
IFS=${WSP_IFS} 


ies 


4 Update list known pair (Address and Name). 
unique lines den pair den pair 
edit exact known pair , den pair 


if [ S(£ den pair[G]) -gt 0 ] # Anything new? 
then 

IFS-$(NO WSP) 

known pair-( ${known_pair[@]} $( den pair[G]) ) 


IFS-S(WSP IFS) 
fi 


# Update list of reverse pairs. 
unique lines den rev den rev 
dit exact reverse pair den rev 


iit [ Sm eem evel sce 0 T f Anything new? 
then 
IFS-S(NO WSP) 
reverse pair-( ${reverse_pair[@]} ${_den_rev[@]} ) 
IFS=${WSP_IFS} 
fa 
# Check indirection limit -- give up if reached. 
ize | eaa _shime= (buen xelods $934 1L) 
then 
return 0 
fi 


# Execution engine is LIFO. Order of pend operations is important. 
# Did we define any new addresses? 


unique lines den address | den address Scrub duplicates. 
edit exact known address | den address Scrub already processed. 
edit exact un address | den address Scrub already waiting. 
if [ $(£ den address[80]) -gt 0 ] Anything new? 
then 

uc address-( S{uc_address[@]} $(. den address[Q]) ) 

pend func expand input address $( den lmt) 


_trace_log[${#_trace_log[@]}]='# Add '$(4 den address[G])' unchkd addr. 


iran 


# Did we find any new names? 


unique lines den nam den nam Scrub duplicates. 
edit exact known name den name Scrub already processed. 
edit exact uc name | den name Scrub already waiting. 
if [ $(£ den name[8]) -gt 0 ] Anything new? 
then 

uc name-( ${uc_name[@]} $( den name[Q]) 

pend func expand input name $( den lmt) 


_trace_log[${#_trace_log[@]}]='#Added '$(4 den name[G])' unchkd name#' 
13 
return 0 


# The parse-it-yourself delegation reply 
4 Input is the chk address list. 

# detail each address «indirection limit» 
detail each address() { 


md 


1485 [ ${#chk_address[@]} -gt 0 ] || return O 


1486 unique lines chk address chk address 

TAST edit_exact known_address chk_address 

1488 if [ ${#chk_address[@]} -gt 0 ] 

1489 then 

1490 known, address-( ${known_address[@]} ${chk_address[@]} ) 
1491 unset chk_address[@] 

1492 ít 

1493 return 0 

1494 } 

1495 

1496 # # # Application specific output functions # # # 
1497 


1498 # Pretty print the known pairs. 
1499 report_pairs() { 


1500 echo 

LSO echo 'Known network pairs. ' 

1502 Coll oriit lanowa poer 2 3. 30) 

1503 

1504 abit [| fracc Chara Eelk exe GO 1 
11505 then 

1506 echo 

1507 echo 'Known chain of authority.' 
1508 Col force utm clara 2 5 30 5S 
1509 í£t3 

1.5110) 

TSEL if [ ${#reverse_pair[@]} -gt 0 ] 
15302 then 

SES echo 

1514 echo 'Known reverse pairs.' 
UHL) GO joresine wewSiese joes 2 5 5S) 
1516 ies 

IESIRI return 0 

TSR Jj 

153.9) 


1520 4 Check an address against the list of blacklist servers. 
1521 # A good place to capture for GraphViz: address-»status (server (reports)) 


1522 4 check lists «ip address» 

1523 Cheek kisra 4 

15:2) S# -eq 1] || return 1 

325 local =a cl, fwd addr 

1525 local -a cl, rev addr 

152 7) Toral =a (eL seeyoldis 

1528 toca =i — (ul rE 

115276) lioxemdl =a, _ Ihe venu 

a local _cl_dns_addr 

ISS logal et dog 

11532 

IDGS golire sus. sil El ve acli . (cul rev pcer 

1534 .Ccl dns addr-$ (dot array . cl rev addr)'.' 

115,55 _ls_cnt=${#list_server[@] } 

1596 echo ' Checking address '$(1) 

qe er Wo etur =e Cee ES eres Silas ))) 

1538 do 

1539 _cl_lkup=${_cl_dns_addr}${list_server[${_cl}]} 

1540 3t imos iced $4 cl Iku} — ell seejolly 

1541 then 

1542 if [ ${#_cl_reply[@]} -gt 0 ] 

1543 then 

1544 echo ' Records from 'S{list_server[${_cl}]} 
1.5415 ACCESS laste lS trackless arcel ON SSL St list eeweset eL 
1546 . hs. RC-2 

1547 gow («V ede = 0 p elle cs Siu el sew s» «eds ) 
1548 do 

1549 echo ' "SA el eee [LS {ellie}; I 


T5510 done 


TOSI iL 


11552 fex 

115/55] done 

1554 return 0 

1555 j 

1556 

1557 4 # # The usual application glue 4 # # 
1:558 


1559 s: Wine nce ane 2 
15600 «uerexeai (0) 1 


ISEL echo 

1562 echo 'Advanced Bash Scripting Guide: is spammer.bash, v2, 2004-msz' 
1563. } 

1564 


1565 4 How to use it? 
1566 4 (See also, "Quickstart" at end of script.) 
1567 usage() ( 


1568 cat ««-' usage statement ' 

1156/9) The script is spammer.bash requires either one or two arguments. 
1570 

115/71 arg 1) May be one of: 

193/27 a) A domain name 

SAS b) An IPv4 address 

1574 C) The name of a file with any mix of names 

LOS and addresses, one per lin 

1576 

LS arg 2) May be one of: 

LSS a) A Blacklist server domain name 

DS b) The name of a file with Blacklist server 

1580 domain names, one per lin 

LSE C) If not present, a default list of (free) 

1.5182 Blacklist servers is used. 

1583 d) If a filename of an empty, readable, file 

1584 is given, 

1585 Blacklist server lookup is disabled. 

1586 

1157 All script output is written to stdout. 

1588 

1559 inNeicucm (exexelexe m 10) =: AT Oi, il Se Serie issues, 

1.5 99) 2 -» Something is Blacklisted. 

1591 

115 9/7 Requires th xternal program 'dig' from the 'bind-9' 
1593 set of DNS programs. See: http://www.isc.org 

1594 

1.5 95; The domain name lookup depth limit defaults to 2 levels. 
1596 Set the environment variable SPAMMER LIMIT to change. 
1597 SPAMMER_LIMIT=0 means 'unlimited' 

1.5 95 

1599) Limit may also be set on the command-line. 

1600 If arg#l is an integer, the limit is set to that value 
1601 and then the above argument rules are applied. 

1602 

1603 Setting the environment variable 'SPAMMER DATA' to a filename 
1604 will cause the script to write a GraphViz graphic file. 
1605 

1606 For the development version; 

1607 Setting the environment variable 'SPAMMER TRACE' to a filename 
1608 will cause the execution engine to log a function call trace. 
1609 

1610 usage statement 

iLodLT. jJ 

1612 


1613 # The default list of Blacklist servers: 

1614 4 Many choices, see: http://www.spews.org/lists.html 
1615 

1616 declare -a default servers 


See: http://www.spamhaus.org (Conservative, well maintained) 
default servers[0]2'sbl-xbl.spamhaus.org' 
See: http://ordb.org (Open mail relays) 
default servers[1]-2'relays.ordb.org' 
See: http://www.spamcop.net/ (You can report spammers here) 
default servers[2]-2'bl.spamcop.net' 
See: http://www.spews.org (An 'early detect' system) 
default servers[3]-2'l2.spews.dnsbl.sorbs.net' 
See: http://www.dnsbl.us.sorbs.net/using.shtml 
default servers[4]-'dnsbl.sorbs.net' 
See: http://dsbl.org/usage (Various mail relay lists) 
default servers[5]-2'list.dsbl.org' 
default servers[6]-'multihop.dsbl.org' 
default servers[7]-2'unconfirmed.dsbl.org' 
# User input argument #1 
setup input() { 
ie [| ee eb |] €s -r ${1} ] # Name of readable fil 
then 
file to array ${1} uc name 
echo 'Using filename >'${1}'< as input.' 
else 
if is address S${1} # IP address? 
then 
Die _dicklzess=( Sb ) 
echo VSrtarcine mito exolebcewye 2: S 1j «v 
else # Must be a name. 
uc name-( ${1} ) 
echo 'Starting with domain name »'$(1)'«' 
ifm 
iB aL 
return 0 
} 
# User input argument #2 
setup servers() { 
ase | =e Sei} | wee se Sq33* ] # Name of a readable file 
then 
file_to_array ${1} list_server 
echo “Usine Ti leneme > STI < as blacklist server lict,” 
else 
list_server=( ${1} ) 
echo 'Using blacklist server »'$(1)'«' 
f 
return 0 
} 
# User environment variable SPAMMER_TRACE 
live_log_die() { 
if [ ${SPAMMER_TRACE:=} ] Wants trace log? 
then 
if [ ! -e ${SPAMMER_TRACE} ] 
then 
if ! touch ${SPAMMER_TRACE} 2>/dev/null 
then 


pend func echo $(printf '%Sq\n' \ 
"Unable to create log file »'$(SPAMM 
pend release 
exit 1 

ia 

.log file-$(SPAMMER TRACE) 

pend hook -trace logger 

.log dump-dump log 

else 
if [ ! -w S(SPAMMER TRACE) ] 
then 


ER TRAC 


(00) TOY OT SS 160) ho 


ie ak 
EL 
return 0 


ít 
.log file-$(SPAMMER TRACE) 
echo 


pen 


pend func echo $(printf '%Sq\n' \ 

"Unable to write log file >'${SPAMMER_TRACE}'<') 
pend release 

eee il 


DMS Sq dog 38 at ley} 
d_hook_=trace_logger 


.log dump-dump log 


# User environment variable SPAMMER DATA 


data capture() { 
if [ S(SPAMMER DATA:-) ] # Wants a data dump? 
then 
if [ ! -e S$(SPAMMER DATA) ] 
then 
if ! touch S$(SPAMMER DATA) 2»/dev/null 
then 
pend func echo $(printf '$q]n' \ 
"Unable to create data output file >'${SPAMMER_DATA}'<') 
pend release 
exit 1 
ít 
dot file-$(SPAMMER DATA) 
.dot dump-dump dot 
else 
if [ ! -w S(SPAMMER DATA) ] 
then 
pend fune echo $(printf '%Sq\n' \ 
"Unable to write data output file »'S$(SPAMMER DATA) '«') 
pend release 
exit 1 
Habd 
dot file-$(SPAMMER DATA) 
.dot dump-dump dot 
fti 


ít 
return O0 


# Grope user specified arguments. 
do user args() { 
if [ $4 -gt Q ] && is number $1 


then 


indirect-$1 


SAIE 


iE aL 


case S# in 


1) 


# Did user treat us well? 


ie | getug imour Sal # Needs error checking. 
then 

pend_release 

$ log dump 

exit 1 
ÍE3E 
list server-( ${default_servers[@]} ) 
_list_cnt=${#list_server[@] } 
echo 'Using default blacklist server list.' 

cho "Seeuecla Cerra limits SA aiachiwescre } 


, 


, 


1749 ise 1 esra noue Si # Needs error checking. 
1750 then 


nari pend release 
i952 $ log dump 
LSS ex di 

1754 fa 

1755 if ! setup servers $2 4 Needs error checking. 
1756 then 

WTS pend_release 
1798 $ log dump 
1759 exea di 

1760 Hisl 

1761 cho 'Search depth limit: '$(indirect) 
1762 "s 

1763 xi) 

1764 pend_func usage 
JS pend release 
1766 $ log dump 

17657 exec di 

1768 s 

1 79 esac 

dL 27/0) return 0 

Ley, 3 

377227 


1773 # A general purpose debug tool. 
1774 4 list array «array name» 
117/75; lier ercran) i 


1776 [ St -eq 1] || return 1 # One argument required. 

JU 

APIS local -a | la lines 

Lyd set -f 

1780 local IFS=${NO_WSP } 

1781 eval ile Iimes=\(\ VE\ISLVTE NIM \) 

1782 echo 

1783 echo "Element count "${#_la_lines[@]}" array "${1} 

1784 local | 1n cnt-$(4 la lines[Q]) 

1785 

1786 icone (Ui sb SO xb =< SML Del wea. e. vebapuE 39] 

1797 do 

1788 exciso Visilenneinc VS at >te Im Jies|[S$ a] 

ISS) done 

EIO set +f 

TPLA return 0 

1792 jy 

1793 

1794 4 # 4 'Hunt the Spammer' program code # 4 # 

TIOS Pene aie # Ready stack engine. 
1796 pend func credits # Last thing to print. 
Hoy 

1798 4 # 4 Deal with user 4 # 4 

1799 live log die # Setup debug trace log. 
1800 data capture # Setup data capture file. 
1801 echo 

1802 do user args $@ 

1803 

1804 4 # 4 Haven't exited yet There is some hope # 4 # 

1805 4 Discovery group - Execution engine is LIFO - pend 

1806 # in reverse order of execution. 

1807 | hs, RC-0 # Hunt the Spammer return code 
1808 pend mark 

1809 pend func report pairs # Report name-address pairs. 
1810 

1811 # The two detail * are mutually recursive functions. 

1812 # They also pend expand * functions as required. 

1813 # These two (the last of ???) exit the recursion. 


1814 pend func detail each address # Get all resources of addresses. 


1815 
1816 
JE y) 
1818 
1819 
1820 
TEZI 
1822 
1823 
1824 
1825 
1826 
15:2; 
1828 
1929 
1830 
SAL 
1832 
LISS 
1834 
LSS, 
1836 
1837 
1838 
19.99) 
1840 
184 
184 
184 
184 
184 
184 
184 
184 
1849 
1850 
1851 
1852 
LISS) 
1854 
1855 
1856 
LSS) 7 
1858 
1L8/5)9) 
1860 
1861 
1862 
1863 
1864 
1865 
1866 
1867 
1868 
1869 
1870 
Ley 
1872 
1873 
1874 
1875 
1876 
ILS 73) 
1878 
1879 
1880 


c -10)015 CO NO S 


pend func detail each name # Get all resources of names. 


# The two expand * are mutually recursive functions, 

#+ which pend additional detail * functions as required. 

pend func expand input address 1 # Expand input names by address. 
pend func expand input name 1 # #xpand input addresses by name. 


# Start with a unique set of names and addresses. 
pend func unique lines uc address uc, address 
pend func unique lines uc name uc, name 


# Separate mixed input of names and addresses. 
pend func split input 


pend release 


# 4 # Pairs reported Unique list of IP addresses found 
echo 


aie 


_ip_cnt=$ {#known_address[@] } 


[ ${#list_server[@]} -eq 0 ] 


then 


echo 'Blacklist server list empty, none checked.' 


else 


iL 


ase [| SH io cat: Seep 0] 
then 
echo 'Known address list empty, none checked.' 
else 
PECENE {ajo puniE }}—il j Sue GNC Es 
echo 'Checking Blacklist servers. ' 
tox (( aj = 319 gmtE gp ajo »— OQ p .cagype- ») 
do 
pend func check lists $( printf '%Sq\n' $(known address[S$ ip]) ) 
done 


EIL 


pend_release 

$ dot dump # Graphics file dump 
Ss) log dump # Execution trace 
echo 


FE AE AE FE AE HHT HH EH HT HH TH AE FE FE AE HE E HE H 


# 


Example output from script # 


HEHEHE HE EERE EE HEE HE EEE EE EE HE HE 
s««-" ss sjoammer EOU PES a 


./is spammer.bash 0 web4.alojamentos7.com 


Starting with domain name >web4.alojamentos7.com< 
Using default blacklist server list. 
Search depth limit: 0 


6.983.203 497 web4.alojamentos7.com. 
G66 S8) a 203. NT nsl.alojamentos7.com. 
(59). 5,5 9 2(02; LAT ns2.alojamentos.ws. 
(96.5 98 c 2/09 .-. 97 alojamentos7.com. 

$$ o 99 « 20/9 o 97 web.alojamentos7.com. 
OSa SE ZOZ LAG nsl.alojamentos.ws. 
69.56.202.146 alojamentos.ws. 
66-235, WSO, 1L1L3 nsl.alojamentos.org. 
$$.5.2 S5 o N81 s 1192 ns2.alojamentos.org. 
EE ASS, AsO). LS: alojamentos.org. 
665235 5180.13 web6.alojamentos.org. 
BAG ASA 2 DARSO nsl.theplanet.com. 

WA 5 MGs LG. 1E 1L ns2.theplanet.com. 


PAG ; 3,955 1L 3L. c HZ maill.theplanet.com. 


1881 69.56.141.4 spooling.theplanet.com. 


1882 PALS «1L 95)» AIL AL, 410) theplanet.com. 

1883 PAL) 5 1195 « JL1L1L 5 440) www.theplanet.com. 
1884 2312 1195 5 131 5 BZ mail.theplanet.com. 
1885 


1886 Checking Blacklist servers. 
1887 Checking address 66.98.208.97 


1888 Records from dnsbl.sorbs.net 
1889 "Spam Received See: http://www.dnsbl.sorbs.net/lookup.shtml1?66.98.208.97" 
1890 Checking address 69.56.202.147 
de Checking address 69.56.202.146 
1892 Checking address 66.235.180.113 
1893 Checking address 66.235.181.192 
1894 Checking address 216.185.111.40 
1895 Checking address 216.234.234.30 
1896 Checking address 12.96.160.115 
d Checking address 216.185.111.52 
1898 Checking address 69.56.141.4 
1899 


1900 Advanced Bash Scripting Guide: is spammer.bash, v2, 2004-msz 
LOG 

1902 _is_spammer_outputs_ 

1903 

1904 exit $( hs RC) 

1905 

1906 HETE A A H TE HE AE HE HE TE FE FE H HE TE FE AE EE TE AE AE EE E TE FE E E E E E E E E E E E E E E E E E E E EEE H 

1907 # The script ignores everything from here on down # 


1908 #+ because of the 'exit' command, just above. # 
1909 Hee ee HE HE HE HE E EEE EE HE HE HEE RE EE E E EE EEE E E EE E E E HE EH 
LOO 

WIL 

LOTZ 

1913 Quickstart 

1914 ========== 

19:15 

1916 Prerequisites 

EOSIN 

1918 Bash version 2.05b or 3.00 (bash --version) 


Sl) A version of Bash which supports arrays. Array 


1920 support is included by default Bash configurations. 
LOZI 
16922 'dig,' version 9.x.x (dig SHOSTNAME, see first line of output) 


1923 A version of dig which supports the +short options. 
1924 See: dig wrappers.bash for details. 


1925 

1926 

1927 Optional Prerequisites 

1928 

119216) 'named,' a local DNS caching program. Any flavor will do. 


1930 Do twice: dig S$HOSTNAME 
TIS, (Clmexels mear borcom (out Ggwe deus c SHANS LAI 1020). WSs) 


1932 That means you have one running. 

1933 

1934 

1935 Optional Graphics Support 

1936 

1937 Volare, a Sibaimeleucel “imix enie Cece =R) 

1938 

1939) dot Program to convert graphic description file toa 


1940 diagram. (dot -V) 

1941 A part of the Graph-Viz set of programs. 

1942 See: [http://www.research.att.com/sw/tools/graphviz||GraphViz] 
1943 

1944 'dotty,' a visual editor for graphic description files. 

1945 Also a part of the Graph-Viz set of programs. 

1946 


1947 
1948 
1949 
1.959 
TOS 
1957 
LISS 
1954 
1959 
1956 
1957 
LOSS 
1956) 
1960 
ocili 
1962 
1963 
1964 
1L9)6/5 
1966 
1967 
1968 
1969 
1970 
JL 97. 
1992 
1973 
1974 
LOWS: 
LD 
LH 
LOVE 
LIS) 
1980 
LOE AL 
1982 
1983 
1984 
19195 
1986 
1987 
1988 
1989 
1.999 
LL 
17992 
Ls) 
1994 
1995 
1996 
199 
LON 
1999 
2000 
2001 
2002 
2003 
2004 
2005 
2006 
2007 
2008 
2009 
2010 
2011 
2012 


Quick Start 


In the same directory as the is spammer.bash script; 
o: ./is spammer.bash 


D 


des 


Ze 


gu 


57 


Usage Details 


Blacklist server choices. 


(m) 9 wwe cCetawle, owi kemiri dits DO motlnuime, 


(IS) WO wise sour guum Juss 


i. Create a file with a single Blacklist server 


domain name 


per line. 


ii. Provide that filename as the last argument to 


the script 


(c) To use a single Blacklist server: 
to the script. 


(d) To disable Blacklist lookups: 


i. Create an empty file (touch spammer.nul) 


Your choice 


ii. Provide th 


of filename. 


Last argument 


filename of that empty file as the 


last argument to the script. 


Search depth limit. 


(a) To use the default value of 2: Do nothing. 


(b) To set a different limit: 


A imie que Q 


means: no limit. 


i. export SPAMMER LIMIT-1 


or whatever 


ii. OR provide the desired limit as the first 


argument to 


(a) To use the default setting of no log output: 


limit you want. 


the script. 


Optional execution trace log. 


(b) To write an execution trace log: 


export SPAMM 


ER TRACE-spammer.log 


or whatever filename you want. 


Optional graphic description file. 


Do nothing. 


(a) To use the default setting of no graphic file: Do nothing. 


(b) To write a Graph-Viz graphic description file: 


export SPAMM 


ER DATA-spammer.dot 


or whatever filename you want. 


Where to start 


(m)) (SiEeueibx S "was 


the search. 


h a single domain name: 


291.3) i. Without a command-line search limit: First 

2014 argument to script. 

2015 

2016 ii. With a command-line search limit: Second 

293. 7 argument to script. 

2018 

291.9) (b) Starting with a single IP address: 

2020 

2021 i. Without a command-line search limit: First 

2027 argument to script. 

2023 

2024 ii. With a command-line search limit: Second 

21025 argument to script. 

2026 

2027 (c) Starting with (mixed) multiple name(s) and/or address(es): 
2028 Create a file with one name or address per line. 
2029 Your choice of filename. 

2030 

2031 i. Without a command-line search limit: Filename as 
2032 first argument to script. 

210.33) 

2034 ii. With a command-line search limit: Filename as 
2035 second argument to script. 

2036 

2037 6. What to do with the display output. 

2038 

2039 (a) To view display output on screen: Do nothing. 
2040 

2041 (b) To save display output to a file: Redirect stdout to a filename. 
2042 

2043 (c) To discard display output: Redirect stdout to /dev/null. 
2044 

2045 7. Temporary end of decision making. 

2046 press RETURN 

2047 wait (optionally, watch the dots and colons). 

2048 

2049 8. Optionally check the return code. 

2050 

AOS (a) Return code 0: All OK 

2052 

2055 (b) Return code 1: Script setup failure 

2054 

2055 (c) Return code 2: Something was blacklisted. 

2056 

2057 9. Where is my graph (diagram) ? 

2058 


2059 The script does not directly produce a graph (diagram). 
2060 It only produces a graphic description file. You can 
2061 process the graphic descriptor file that was output 
2062 meea icine "glo" jxeegnesumns 

2063 

2064 Until you edit that descriptor file, to describe th 
2065 relationships you want shown, all that you will get is 
2066 a bunch of labeled name and address nodes. 

2067 

2068 All of the script's discovered relationships are within 
2069 a comment block in the graphic descriptor file, each 
2070 with a descriptive heading. 

ZO 11. 

2072 The editing required to draw a line between a pair of 
2073 nodes from the information in the descriptor file may 
2074 be done with a text editor. 

2075 

2076 Given these lines somewhere in the descriptor file: 
2077 

2078 4 Known domain name nodes 


2080 NO000 [label="guardproof.info."] 


2082 N0002 [label="third.guardproof.info."] ; 


2086 # Known address nodes 


2096 A0000 [leisels wsi Ai 34.19! || 


2092 


2094 # Known name->address edges 


AVDS nwew9 99 Tai e etenn .aAMIEO, G A 32.197 


2099 

2100 # Known parent-»child edges 

LoL 

102 PC0000 guardproof.info. third.guardproof.info. 
103 

diga =y 

105 

106 Turn that into the following lines by substituting node 
107 identifiers into the relationships: 

108 

109 # Known domain name nodes 

110 

111 NOOOO [label="guardproof.info."] 

LLZ 

113 N0002 [label="third.guardproof.info."] 

114 

ALS) 

116 

117 # Known address nodes 

118 

119 A0000 [label="61.141.32.197"] 

120 

T21 

122 

123 # PC0000 guardproof.info. third.guardproof.info. 
124 

125 N0000-»N0002 ; 

126 

1279 

128 

129 e IUNOOOO) chairo ejuieiechoseOoiE o dmt Ob, ISZ. 197 
130 

131 N0002-»A0000 ; 

1192] 

133 

134 

195 7 

136 

137 # Known name-»address edges 

138 

T39 MANCOOO tia rei eieaa rooks ae Gil, IAL 32 « 197) 
140 

141 

142 

143 # Known parent-»child edges 

144 


KE INSECTS dS INSETS [SS TSS TOE TSS TNS) IRS INGE INS TSS) ISS) TSOP TSS TNS) [SS SSE INST ISS) SY TSE TSOP TNS) fS INSETS TSS) ISS) TSE TSS TN) IRS) fS) SS INS) ISS TOE TSE TNS) ISS) 


145 
146 
147 
148 
149 
L50 
LSA 
152 
155 
154 
4155 
156 
157 
158 
159 
160 
161 
162 
LOS} 
164 
LOS 
166 
16:7 
168 
169 
4L 7/0) 
JL iL 
172 
Lys} 
174 


[NOISY TSF TSS ASS) ISS) [ROP INS TROY TS O TSS) ISS) INST TROP INS ISS) ISS) [SO TRS) INS) ISP USP INS ISP ISS) TO INS) INS) NS) 


PC0000 guardproof.info. third.guardproof.info. 
iA 


Process that with the 'dot' program, and you have your 
first network diagram. 


In addition to the conventional graphic edges, the 
descriptor file includes similar format pair-data that 
describes services, zone records (sub-graphs?), 
blacklisted addresses, and other things which might be 
interesting to include in your graph. This additional 
information could be displayed as different node 
shapes, colors, line sizes, etc. 


The descriptor file can also be read and edited by a 
Bash script (of course). You should be able to find 
most of the functions required within the 

"is spammer.bash" script. 


# End Quickstart. 


Additional Note 


Michael Zick points out that there is a "makeviz.bash" interactive 
Web site at rediris.es. Can't give the full URL, since this is not 
a publically accessible site. 


Another anti-spam script. 


Example A-29. Spammer Hunt 


NPRPRPPRP PPP PY 
O (o 00 -1 O Ui i$ (). NM P O xo 0 -1 O O1 4 CQ I EO 


NNNNNN ND 
sl] py Gal des (G8) IS) J 


!/bin/bash 

whx.sh: "whois" spammer lookup 

Author: Walter Dnes 

Slight revisions (first section) by ABS Guide author. 
Used in ABS Guide with permission. 


Needs version 3.x or greater of Bash to run (because of =~ operator). 
Commented by script author and ABS Guide author. 


E_BADARGS=65 Missing command-line arg. 
E_NOHOST=66 Post ione se ojubavel 
E TIMEOUT-67 Host lookup timed out. 


# 
# 
# 
E UNDEF=68 # Some other (undefined) TEO o 
# 
# 
# 


HOSTWAIT=10 Specify up to 10 seconds for host query reply. 
The actual wait may be a bit longer. 


whois.txt Output file. 


ILIE Sa Visit j| # Check for (required) command-line arg. 
then 

echo "Usage: $0 domain name or IP address" 

exit SE_BADARGS 


28 
29 
30 
Sil 
32 
ES 
34 
33 
36 
SY 
38 
3g 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Sl 
52 
53) 
54 
939 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
y 
72 
3:3 
74 
US 
76 
GF 
78 
1 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
9i 
92 
9S) 


if [[ "S1" ~ "[a-zA-Z][a-zA-Z]S" |] # 
then ü 


IPADDR=S (host -W SHOSTWAIT $1 | awk ' 
# 
d 

else 

IPADDR-"$1" # 
ifaL 
Gehoor echo Wine Welchesss: asia US IPIE Ws 
ae [| Ses VSOWWE III ] 
then 

ie cie SOUT ILI, 

echo "Sil (wee rile  WUUSXOIUHDS JE IIS 
Fi 

Sanity checks. 
(This section needs more work.) 

if [ -z "SIPADDR" ] 

No response. 
then 

echo "Host not found!" 

exit SE_NOHOST # Bail out. 
Ieai 
i [I “SIPADDRY Sx V^] T] 
# 37; connection timed out; no servers 
then 

echo "Host lookup timed out!" 

exit $E TIMEOUT BaT lL serene ¢ 
ft 
ae || SIP ANDIDIRY = |) NEOA S" 3T 
io HOSE OOXESSQOXNONNONG RG do KonE EOIR 


then 


Ends in two alpha chars? 

It's a domain name && must do host lookup. 
Doriar Sp") 

Doing host lookup to get IP address. 


japeinieevere Eamel deseo. 


Command-line arg was IP address. 


echo 


removed."; echo 


could be reached 


3 (NXDOMAIN) 


Seino Wiskosne MOL sec 

COSNMESIMESNOEHGO SI # Bail out. 
al 
mae [pp USED). e RESERVERAT S] 
HOSEA. MOK NOE 3t OLa - 22 (SUT IRWWAD VAI IL) 
then 

echo "Host not found!" 

SSeS INOW OSL # Bail out. 
I3 
# Main body of script 
AFRINICquery() { 
# Define the function that queries AFRINIC. Echo a notification to the 


#+ screen, and then run the actual quer 


echo "Searching for SIPADDR in whois. 
whois -h whois.afrinic.net "SIPADDR" 


# Check for presence of reference to a 
# Warn about non-functional rwhois.inf 
#+ and attempt rwhois query. 

aie Cae -e Yess giclee. [L^ 


1\ 


ie 


Wp weelireeveiestinsy; Owlirjsuie TO SOUT WL, 


afrinic.net" 
= GOUIE INE 


n rwhois. 
DSAL noel SOPVOT 


ar SOULE TIED 


94 


jas 
«o 


AP 


then 
exo W " xx SOUTE LLE 
SCO Wester x» SO ae IIL 
acho Were Ss: SOUTE LLE 
echo "Warning: rwhois.infosat.net was not working as of 2005/02/02" >> SOUTFIL 
echo " when this script was written." >> SOUTFILE 
SENO Were x SO are IIb 
acho Weel Ss SOMME ILS 
echo " " >> SOUTFILE 
RWONS=" Orao VMesmackss s Anos A Jew VOUER | tarir -m a [X 
Seci Ui Xs A noe NA a S) NDS 
whois -h ${RWHOIS}:${PORT} "SIPADDR" >> SOUTFILI 

EA 


EI 


imd 


NICquery() { 
echo "Searching for SIPADDR in whois.apnic.net" 
whois -h whois.apnic.net "SIPADDR" > SOUTFILE 


Just about every country has its own internet registrar. 

I don't normally bother consulting them, because the regional registry 
usually supplies sufficient information. 

There are a few exceptions, where the regional registry simply 

refers to the national registry for direct data. 

These are Japan and South Korea in APNIC, and Brasil in LACNIC. 

The following if statement checks SOUTFILE (whois.txt) for the presence 
of "KR" (South Korea) or "JP" (Japan) in the country field. 

If either is found, the query is re-run against the appropriate 
national registry. 


ie Gace c Wwexexbbsuexewz e [L aS WS Owais IIL 
then 
echo "Searching for S$IPADDR in whois.krnic.ne 
whois -h whois.krnic.net "SIPADDR" >> SOUTFIL 
elir grep = V"^egewswErys| ls" VSQVE TIU 
then 
echo "Searching for SIPADDR in whois.nic.ad. jp" 
whois -h whois.nic.ad.jp "SIPADDR"/e >> SOUTFIL 
E 


152 EN gs 


pe 


ARINquery() { 


@ 
@ 


echo "Searching for SIPADDR in whois.arin.net" 
whois -h whois.arin.net "SIPADDR" > SOUTFILE 


Several large internet providers listed by ARIN have their own 
internal whois service, referred to as "rwhois". 

A large block of IP addresses is listed with the provider 

under the ARIN registry. 

To get the IP addresses of 2nd-level ISPs or other large customers, 
one has to refer to the rwhois server on port 4321. 

I originally started with a bunch of "if" statements checking for 
the larger providers. 

This approach is unwieldy, and there's always another rwhois server 
that I didn't know about. 

A more elegant approach is to check $OUTFILE for a reference 

to a whois server, parse that server name out of the comment section, 
and re-run the query against the appropriate rwhois server. 

The parsing looks a bit ugly, with a long continued line insid 
backticks. 
But it only has to be done once, and will work as new servers are added. 

ABS Guide author comment: it isn't all that ugly, and is, in fact, 

+ an instructive use of Regular Expressions. 


Lit Grop -E MACemmcines woe. [p^ Tr “Sowa CIUS 
then 


MNONNNNNN PS 


N 
Ko) 


RWHOIS= grep -e "^Comment:.*rwhoisN.[^ ]\+" "SOUTFILE" 


Sel SIEEN (5 AN Nana S [> INEM) ENN ANA 

echo "Searching for S$IPADDR in ${RWHOIS}" 

whois -h ${RWHOIS}:${PORT} "SIPADDR" >> SOUTFILI 
if aL 


Lu 


LACNICquery() { 


# 
#+ 
# 


RI 


sl 


sl 
# 


oc 


E 


el 


el 


el 


echo "Searching for S$IPADDR in whois.lacnic.net" 
whois -h whois.lacnic.net "SIPADDR" > SOUTFILE 


The following if statement checks SOUTFILE (whois.txt) 


"BR" (Brasil) in the country field. 


| tait =m LÑ 


for the presence of 


If it is found, the query is re-run against whois.registro.br. 


Le grep =a VAcoumemys sse USO 
then 
echo "Searching for SIPADDR in whois.registro.br" 
whois -h whois.registro.br "SIPADDR" >> SOUTFILE 
fta 


PEquery() { 
echo "Searching for SIPADDR in whois.ripe.net" 
whois -h whois.ripe.net "SIPADDR" > SOUTFILE 


Initialize a few variables. 

* slash8 is the most significant octet 

* slashl6 consists of the two most significant octets 
* octet2 is the second most significant octet 


esie SElno; STADDIR || elite —cl, =i d 


if [ -z "Sslash8" ] 4 Yet another sanity check. 
then 

echo "Undefined error!" 

exit SE_UNDEF 
Teal 
ashl6-'echo SIPADDR | cut -d. -f 1-2^ 

^ Period specified as 'cut" 

aie p c2 Uegbewsnd | 
then 

echo "Undefined error!" 

exit $E UNDEF 
ical 
tet2= echo Sslashl6 | cut -d. -f 2^ 
ie | =a USeOcue2" ] 
then 

echo "Undefined error!" 

exit SE_UNDEF 


i aL 


Check for various odds and ends of reserved space. 
There is no point in querying for those addresses. 


[ S$slash8 == 0 ]; then 
echo SIPADDR is '"This Network"' space\; Not querying 
if [ Sslash8 == 10 ]; then 
echo SIPADDR is RFC1918 space\; Not querying 
if [ Sslash8 == 14 ]; then 


delimiter. 


echo SIPADDR is '"Public Data Network"' space\; Not querying 


abe | Scola slats: SS IDI d E. ehen 
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Day 
228 
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230 
2.34 
232 
23.3 
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235 
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23 
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echo SIPADDR is loopback space\; Not querying 
NEG $slashl16 == 169.254 ]; then 
echo SIPADDR is link-local space\; Not querying 
elif $slash8 -- 172 ] && Soctet2 -ge 16 ] && [ Soctet2 -le 31 ];then 
echo SIPADDR is RFC1918 space\; Not querying 
alii $slashl16 == 192.168 ]; then 
echo SIPADDR is RFC1918 space\; Not querying 
elif Sslash8 -ge 224 ]; then 
echo SIPADDR is either Multicast or reserved space\; Not querying 
alii $slash8 -ge 200 ] && [ $slash8 -le 201 ]; then LACNICquery "SIPADDR" 
elif $slash8 -ge 202 ] && [ $slash8 -le 203 ]; then APNICquery "SIPADDR" 
elif $slash8 -ge 210 | && [ $slash8 -le 211 ]; then APNICquery "SIPADDR" 
elir $slash8 -ge 218 ] && [ $slash8 -le 223 ]; then APNICquery "SIPADDR" 
# If we got this far without making a decision, query ARIN. 
# If a reference is found in SOUTFILE to APNIC, AFRINIC, LACNIC, or RIPE, 
#+ query the appropriate whois server. 
else 
ARINquery "SIPADDR" 
if grep "whois.afrinic.net" "SOUTFILE"; then 
AFRINICquery "SIPADDR" 
elit grep -5 W OrgiDs | PRESS% ET SOUDESISE then 
RIPEquery "SIPADDR" 
elif grep -E "^OrgID:| ]+APNICS" "SOUTFILE"; then 
APNICquery "SIPADDR" 
elif grep -E "^OrgID:[ ]+LACNICS" "SOUTFILE"; then 
LACNICquery "SIPADDR" 
IE al 
EL 
@ 
Tey also: 
wget http://logi.cc/nw/whois.php3?ACTION-doQuery&DOMAIN-S$IPADDR 
@ 
We've now finished the querying. 
Echo a copy of the final result to the screen. 
cat SOUTFILE 
Or Wess POUT NH 
exit 0 
@ ABS Guide author comments: 
@ Nothing fancy here, but still a very useful tool for hunting spammers. 
@ Sure, the script can be cleaned up some, and it's still a bit buggy, 
@+ (exercise for reader), but all the same, it's a nice piece of coding 
@+ by Walter Dnes. 
@ Thank you! 


"Little Monster's" front end to wget. 


Example A-30. Making wget easier to use 
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#!/bin/bash 


# wgetter2.bash 


Author: 


Se SH che cR 


Little Monster 
==> Used in ABS Guide with permission of script author. 
==> This script still needs debugging and fixups ( 
==> It could also use some additional editing in the comments. 


[monster@monstruum.co.uk] 


xercise for reader). 
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* email me at 
==> and cc: 
WAALS Swe 


This is wgetter2 -- 
+ a Bash script to make wget a bit more friendly, and save typing. 


Carefully crafted by Little Monster. 
More or less complete on 02/02/2005. 
If you think this script can be improved, 


monster@monstruum.co.uk 


to the author of the ABS Guide, please. 


is licenced under the GPL. 


YOU are free CO Copy; alter mmol ie-use itp 
+ but please don't try to claim you wrote it. 
Log your changes here instead. 


changelog: 


07/02/2005. 
02/02/2005. 


29/01/2005. 
22/11/2004. 
01/12/2004. 
01/12/2004. 
01/12/2004. 
01/12/2004. 


01/12/2004. 


05/12/2004. 


01/02/2004. 


Fixups by Little Monster. 

Minor additions by Little Monster. 

(See after # +++++++++++ ) 

Minor stylistic edits and cleanups by author of ABS Guide. 
Added exit error codes. 

Finished initial version of second version of wgetter: 
wgetter2 is born. 

Changed 'runn' function so it can be run 2 ways -- 
either ask for a file name or have one input on the CL. 
Made sensible handling of no URL's given. 

Made loop of main options, so you don't 

have to keep calling wgetter 2 all the time. 

Runs as a session instead. 

Added looping to 'runn' function. 

Simplified and improved. 

Added state to recursion setting. 

Enables re-use of previous value. 

Modified the file detection routine in the 'runn' function 
So it's not fooled by empty values, and is cleaner. 
Added cookie finding routine from later version (which 
isn't ready yet), so as not to have hard-coded paths. 
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_ USAGE- 67 
ONO (Ole S,—(69 
NO. URLS-69 


El E p 


T 


| USER, EXIT-71 


This is the 


. NO. SAVEFILE-70 


Error codes for abnormal exit. 


# Usage message, then quit. 

No command-line args entered. 

No URLs passed to script. 

No save filename passed to script. 
User decides to quit. 


# 
# 
# 
# 


Basic default wget command we want to use. 


place to change it, if required. 


NB: if using a proxy, set http_proxy = yourproxy in .wgetrc. 


Otherwise delet proxy=on, below. 


CommandA="wget 


TNC Che Ute) progress=bar random-wait proxy-on -r" 


# 
# Set some other variables and explain them. 
pattern=" -A .jpg, .JPG, .jpeg, .JPEG, .gif£, .GiF, .htm, .html, .shtml, .php" 


# wget's option to only get certain types of file. 
# comment out if not using 


today= date -*$F' # Used for a filename. 


PRPRPPPrPP RB 
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home=$HOME Set HOME to an internal variable. 

In case some other path is used, change it here. 
depthDefault=3 Set a sensible default recursion. 
Depth=SdepthDefault Otherwise user feedback doesn't tie in properly. 
RefA="" Set blank referring page. 

Flag-"" Default to not saving anything, 

+ or whatever else might be wanted in future. 
lister="" Used for passing a list of urls directly to wget. 
Woptions="" Used for passing wget some options for itself. 
inFile="" Used for the run function. 
newFile="" Used for the run function. 


savePath-"$home/w-save" 
Config="Shome/.wgetter2rc" 
This is where some variables can be stored, 
+ if permanently changed from within the script. 
Cookies liie" Simone ,cookielasic™ 


So we know where the cookies are kept 
cFlag-"" Part of the cookie file selection routine. 
# Define the options available. Easy to change letters here if needed. 
# These are the optional options; you don't just wait to be asked. 
save=s Save command instead of executing it. 
cook=c Change cookie file for this session. 
help=h Usage guide. 
list-l Pass wget the -i option and URL list. 
runn-r Run saved commands as an argument to the option. 
inpu-i Run saved commands interactively. 
wopt=w Allow to enter options to pass directly to wget. 
# 
ae (p =e WW EEn # Make sure we get something for wget to eat. 
echo "You must at least enter a URL or option!" 
echo Wolle. ror Tage,” 
exit $E_NO_OPTS 
ic 
E RH E RHET RE RM TE A HH TTE TEE EE ATA EET ATE EE B EE YR TEE EAE BBMAEE E | MeBREREE 
# added added added added added added added added added added added added 
mie dp Jg =e Vsicomirne’ Tg. elaein # See if configuration file exists. 
echo "Creating configuration file, $Config" 


echo "f This is the configuration file for wgetter2" > "SConfig" 


echo "£f Your customised settings will be saved in this file" >> "$Config" 


else 

source $Config # Import variables we set outside the script. 
EL 
ISF ! —-e "SCookie List" ]; then 


Set up a list of cookie files, if there isn't one. 
Selo ViBiokawEdLUmgy itus COOKIES s s V 


ít 3b Isolate this in its own 'if' statement, 
+ in case we got interrupted while searching. 


E Sa Vgl Jg ftem If we haven't already done this 
echo Make a nice space after the command prompt. 
echo "Looks like you haven't set up your source of cookies yet." 
n=0 Make sure the counter 


+ doesn't contain random values. 

while read; do 
Cookies [$n] =SREPLY Put the cookie files we found into an array. 
echo "$n) $(Cookies[$n])" # Create a menu. 


find -name cookies.txt >> SCookie List # Create the list of cookie files. 
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mega ++ I jy) # Increment the counter. 
done « $Cookie List # Feed the read statement. 
echo "Enter the number of the cookie file you want to use." 
echo "If you won't be using cookies, just press RETURN." 
echo 
echo "I won't be asking this again. Edit $Config" 
echo "If you decide to change at a later date" 
echo "or use the -$(cook) option for per session changes." 
read 
ar [| | =i Sume Jo ENEN # User didn't just press return. 
Gook i e= load-cookies $(Cookies[SREPLY])" 
# Set the variable here as well as in the config file. 


eho Cookies! load-cookies ${Cookies[$REPLY]}\"" >> $Config 
Fi 
echo "cFlag-1" >> $Config # So we know not to ask again. 
ia 


# end added section end added section end added section end added section 
E RERERRRR EE EBAY TT4 YF B AE B AE BAMA- BAAA B YEFRY.TBBEET A.B BEBE. AYAGMY 


# Another variable. 
# This one may or may not be subject to variation. 
# A bit like the small print. 
CookiesON-$Cookie 
# echo "cookie file is $CookiesON" # For debugging. 
# echo "home is $(home)" # For debugging. 
4 Got caught with this one! 


wopts () 

{ 

echo "Enter options to pass to wget." 

echo "It is assumed you know what you're doing." 

echo 

echo "You can pass their arguments here too." 

That is to say, everything passed here is passed to wget. 


read Wopts 
Read in the options to be passed to wget. 


Woptions-" $Wopts" 

^ Why the leading space? 
Assign to another variable. 
Just for fun, or something 


echo "passing options $(Wopts) to wget" 
Mainly for debugging. 
Te (ues 


return 


) 


save func() 


{ 


echo "Settings will be saved." 


sae [p d =el SewhweisehEl. |7 then 4 Xe cus huecos eders, 
mkdir $savePath # Create the directory to save things in 
sie abun Ap. shige Vie alreci Chers, 
fti 
Flag=S 


i? Reli rie enaa Jone, «out Ce miet co Clo. 
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# Set a flag since stuff is done in main. 


TOTUN 

} 

usage() # Tell them how it works. 

{ 

cho "Welcome to wgetter. This is a front end to wget." 

echo "It will always run wget with these options:" 
echo "SCommandA" 
echo "and the pattern to match: $pattern \ 

(which you can change at the top of this script)." 
echo "It will also ask you for recursion depth, \ 

and if you want to use a referring page." 


echo "Wgetter accepts the following options:" 


echo "" 
echo "-Shelp : Display this help." 
cho "-$save : Save the command to a file $savePath/wget-($today) \ 

IIMSIESEIC! (gir Ave ae, A 

echo "-$runn : Run saved wget commands instead of starting a new one -" 

echo "Enter filename as argument to this option." 

echo "-Sinpu : Run saved wget commands interactively --" 

echo "The script will ask you for the filename." 

echo "-$cook : Change the cookies file for this session." 


echo "-$1ist : Tell wget to use URL's from a list instead of \ 
from the command-line." 


echo "-$wopt : Pass any other options direct to wget." 
echo "wn 
emo VS the wget man page for additional options \ 

you can pass to wget." 

echo "nn 

exit $E USAGE # End here. Don't process anything else. 
} 
list func() # Gives the user the option to use the -i option to wget, 

#+ and a list of URLs. 

( 
while [ 1 ]; do 


echo "Enter the name of the file containing URL's (press q to change 
youre mael) o 


read urlfile 
a [ d ee USwselsrape qp Gus qp Stes e xp Ie EAE 
i7 bOO ito ev cila, Or Cne Ciklulic OTL 
echo "That file does not exist!" 
exse || Spreken iet = ep iF telaein xp (elvexels CE ERO 
Sche Viro teag a wiell Misto” 
return 
else 
Gelne insiste; Sur liila% 
echo "If you gave url's on the command-line, I'll use those first." 
# Report wget standard behaviour to the user. 
lister-" -i Surlfile" # This is what we want to pass to wget. 
Totus 
fi 
done 
} 
cookie_func() # Give the user the option to use a different cookie file. 
{ 
wüesLlke I dE je clo 


echo "Change the cookies file. Press return if you don't want to change 


(99. Gy) (v9) (63) (93). Wey es) (69) 


WW 
IY 4; 
(e) We) 


ages 
read Cookies 
# NB: this is not the same as Cookie, earlier. 
# There is an 's' on the end. 
4 Bit like chocolate chips. 
3f [ =z "SCooksyes" ]; then # Escape clause for wusses. 
TOL 
elus | I ese "SCcookies" p rhen 
echo "File does not exist. Try again." # Keep em going 
else 
CookiesON-" load-cookies $Cookies" # File is good -- use it! 
Tol DA 
ia 
done 
} 
u cne (() 
{ 
if [ -z “SOPTARG" ]; then 
# Test to s if we used the in-line option or the query one. 
a [ 2 eg VWSseweietim" Te aeu # If directory doesn't exist 
echo "$savePath does not appear to exist." 
echo "Please supply path and filename of saved wget commands:" 
read newFile 
until [ -f "SnewFile" ]; do # Keep going till we get something. 
echo "Sorry, that file does not exist. Please try again." 
# Try really hard to get something. 
read newFile 
done 
se || = (( Ges were Senici lel Y dg chen 
# Assume they haven't got the right file and bail out. 
echo "Sorry, that file does not contain wget commands.  Aborting." 
exit 
fi 
This is bogus code. 
It doesn't actually work. 
If anyone wants to fix it, feel free! 
filePath="${newFile}" 
else 
echo "Save path is SsavePath" 
echo "Pleas nter name of the file which you want to use." 
echo "You have a choice of:" 
ls $savePath # Give them a choice. 
read inFile 
until [ -f "$savePath/SinFile" ]; do # Keep going till 
#+ we get something. 
aoe (p d = US ea VORE e aaewrsblbe e enean 3» ise desLlke: CESSE (She s 
echo "Sorry, that file does not exist. Please choose from:" 
ls $savePath # If a mistake is made. 
read inFile 
ie 
done 
filePath="${savePath}/S${inFile}" # Make one variable 
ít 
lse filePath="${savePath}/${OPTARG}" # Which can be many things 
ie aL 
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aa | do "SuxlebPesim" Ip iem 


echo "You did not specify a suitable file." 
GENO WRU Ele: Serio meita dele Sleeve) GITON ELESTA 
echo "Aborting." 
exit SE NO SAVEFILE 
EL 
echo "Using: $filePath" 


while read; do 
eval SREPLY 
echo "Completed: SREPLY" 


# If a bogus file got through. 


done < SfilePath # Feed the actual file we are using into a 'while' loop. 
exit 
} 
# Fish out any options we are using for the script. 
# This is based on the demo in "Learning The Bash Shell" (O'Reilly). 
while getopts ":SsaveScookShelpS$listSrunn:S$inpuS$wopt" opt 
do 
case Sopt in 
Ssave) save_func;; Save some wgetter sessions for later. 
Scook) cookie_func;; Change cookie file. 
Shelp) usage;; Get help. 
Slabsnr). Isl sie _sruiner. 5 Allow wget to use a list of URLs. 
Sewon) Ie 1E baWep p Useful if you are calling wgetter from, 
T ikon (sxeWslier 5 ici Ger. 
SLND) sau’ 5 When you don't know what your files are named. 
Swopt) wopts;; Pass options directly to wget. 
X2) echo Vio e valid (raa c 4 
echo "Use -S{wopt} to pass options directly to wget," 
seno "owe —(S4lwedjs] sow JaelggUp E # Catch anything else. 
esac 
done 
shift S((OPTIND - 1)) # Do funky magic stuff with $#. 
sie D Sv wed Cows [Poa edsteürcexew 6 ie laveia 
# We should be left with at least one URL 
#+ on the command-line, unless a list is 
#+ being used -- catch empty CL's. 
echo "No URL's given! You must enter them on the same line as wgetter2." 
echo "E.g.,  wgetter2 http://somesite http://anothersite." 
echo "Use $help option for more information." 
exit $E NO URLS # Bail out, with appropriate error code. 
ie iL 
URLS-" AON 
# Use this so that URL list can be changed if we stay in the option loop. 


wikis, || iL d p ce) 
# This is where we ask for the most used options. 
# (Mostly unchanged from version 1 of wgetter) 


Lit || mxemueibxeys"rim |A wlan 
Current="" 
lse Current=" Current value is $curDepth" 
fu 


echo "How deep should I go? \ 

(integer: Default is S$depthDefault.$Current)" 
read Depth # Recursion -- how far should we go? 
inputB="" # Reset this to blank on each pass of the loop. 
echo "Enter the name of the referring page (default is none). 
read inputB # Need this for some sites. 


echo "Do you want to have the output logged to the terminal" 


[Es (SS dis us des dus ES ges e des Es [sS des qfes ES ES des es qi» es des pes qiEy qus ges des We. Jes des WES qus des des des ey Jes gEs. des quy Qu. des des des qus. des. Hus qus qus pes des dE qim des. dus qus qs SS ge ges 
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COMO BNO MINS des» 
[— €» Wer (o5 cl 5 


[99 
N 


C0 CO CO CO CO CO CO 
$0 0 -—-1 0) O1 Bf Ww 


QP fF PP SP SP SS aus 
(Sy ey (es) « ey Gl des (65 fu» [3 m 


or Ol 
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(Gm (On (i 
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57 


Copy (63 LON} (en ion- (env Xen» <Ony Con) Kony (nk (On 
Wey (99) 3] py (Onl des (ES. IS) [— ey Wey (o9) 


echo 


"(s/mw cerade 3e yes) 2! 


read noHide # Otherwise wget will just log it to a file. 


case 


y 
n 


esac 


abit [ 
i uU 
#+ 1 

3b 


# 


SnoHide in # Now you see me, now you don't. 
X tdeo 

l|. Hirden ED; 

A Pide = uT 

-z S(Depth) ]; then 

Ser accepted either default or current depth, 


n which case Depth is now empty. 

i [ =4 Stenna) je ien 
See if a depth was set on a previous iteration. 
Depth="S$depthDefault" 
# Set the default recursion depth if nothing 
#+ else to use. 


lse Depth-"$curDepth" # Otherwise, set the one we used befor 


iB aL 
TEL 
Recurse-" -] $Depth" # Set how deep we want to go. 
curDepth-$Depth # Remember setting for next time. 
ait || I ex SunegxWE lg (xem 
RefA=" referer-$inputB" # Option to use referring page. 
iat 
WGETTER="S$ {CommandA}${pattern}${hide}${RefA}${Recurse}\ 


${CookiesON}${lister}${Woptions}${URLS}" 
# Just string the whole lot together 
# NB: n 
# They are in the individual elements so that if any are empty, 
#+ we don't get an extra space. 


a [ 


EI 


at [ 


e 


=Z 
echo 
# T 
#+ i 


USE 
cho 
Giz 
cho 
Ma 
dS 
cho 


cno 


o embedded spaces. 


VS exe I me [p Weeds = Wily JL p queue 
"AgNeWewLig == (exeuar/ (p iesus COS itae 

his should be changed, 

n case the user has opted to not use cookies. 


agr s SS o2 Chen 

"SWGETTER" >> S$savePath/wget-S$(today) 

eate a unique filename for today, or append to it if it exists. 
"SinputB" >> S$savePath/site-list-$[(today) 

ke a list, so it's easy to refer back to, 

nce the whole command is a bit confusing to look at. 

"Command saved to the file $savePath/wget-S$(today)" 

# Tell the user. 

"Referring page URL saved to the files \ 


savePath/site-list-$(today)" 


# Tell the user. 


Savers" with save option" 
# Stick this somewhere, so it appears in the loop if set. 
else 
echo Wok Ck ck kk ck kck ck ck kck k kc k " 
exeliyg. Wess er so(Greib ibaigj o etta 
echo Wok ok ck ck ck ckockckck ck ck ck k kk M 
echo "" 
echo "SWGETTER" 
echomu 
echo Wok Ck ck ck ck kock kckockckck k kc k M" 
eval "SWGETTER" 
ít 
Sien M 
echo "Starting over$Saver." 


470 echo "If you want to stop, press q." 

AT echo "Otherwise, nter some URL's:" 

472 # Let them go again. Tell about save option being set. 
473 

474 read 

475 case SREPLY in 

476 # Need to change this to a 'trap' clause. 

Arq) eG 3 exit Sm USRR KIA p i ldbseucule(e Or (cls reacer? 
478 * ) URLS-" SREPLY";; 

a) esac 

480 

481 acino wu 

482 done 

483 

484 

485 exit 0 


Example A-31. A podcasting script 


!/bin/bash 


bashpodder.sh: 
By bine 10/1/2004 
Find the latest script at 
* http://linc.homeunix.org:8080/scripts/bashpodder 
Last revision 12/14/2004 - Many Contributors! 
If you use this and have made improvements or have comments 
+ drop me an email at linc dot fessenden at gmail dot com 
I'd appreciate it! 


==> ABS Guide extra comments. 


==> PRT EEE HHH HHH THEE HHH EH HT HEH HE EE EEE aE 


Mop pop pp PPP PY 
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==> What is "podcasting"? 


24l 

22 ==> It's broadcasting "radio shows" over the Internet. 

29 ==> These shows can be played on iPods and other music file players. 
24 

ZS) ==> This script makes it possible. 

26 ==> ŞS documentation at the script author's site, above. 

21 

28 ==> E E E AE E EE E FE EE HEE HE HEHEHE EE HE HE EEE EEE EE HE EERE EE E E E HE HEE E E EE HH EH 


Make script crontab friendly: 
cd $(dirname $0) 
==> Change to directory where this script lives. 


datadir is the directory you want podcasts saved to: 
datadir-$ (date +%Y-%m-%d) 
==> Will create a date-labeled directory, named: YYYY-MM-DD 


CO CO CO CO CO CO CO CO CO CO PO 
«OC a a I OY CH HS OO LON ESO COS Ke) 


Check for and create datadir if necessary: 


40 if test ! -d Sdatadir 
A then 
42 mkdir $datadir 


43 fi 


45 


# Delete any temp fil 


rm —f temp.log 


# Read the bp.conf file and wget any url not already 
#+ in the podcast.log file: 
while read podcast 
do 4 ==> Main action follows. 
iale-S(wertE C Sees —9 = | wie V Ne" Nm! [D sexe \ Ww [Do 
sech =a Ve eel C SNO) ur cte TE IA p 
fOr Wiel aim SitLile 
do 
echo $url >> temp.log 
xit | gesp Youri wecessic.log > /clew/muwilil 


then 
wget € =P Sdatadict “Suri” 


ie al 
done 
done < bp.conf 


# Move dynamically created log file to permanent log file: 
cat podcast.log »» temp.log 
SOre cem log | wine > joocleasic leg 


rm temp.log 


# Create an m3u playlist: 
| grep -v m3u > $datadir/podcast.m3u 


iiss ‘Siclevt eyelat ie 


FEFE TE TE TE FE FE FE FE FE TE FE FE FE FE FE FE FE FE FE FE FE FE FE FE EEE HE EEE HE EEE HEHE HHH HHH EH 
For a different scripting approach to Podcasting, 
see Phil Salkie's article, 

"Internet Radio to Podcast with Shell Tools" 

in the September, 2005 issue of LINUX JOURNAL, 
http://www. linuxjournal.com/article/8171 
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Example A-32. Nightly backup to a firewire HD 


(ee) c] (x, Crh dew. GS) DS) I) (c9. Wer (ee) c E Gy al des GO f$. [3 


NN DN + 
IS) [3 eS) We) 


!/bin/bash 


Copyright 


Save as: 


nightly-backup.sh 
http://www.richardneill.org/source.phpfknightly-backup-rsync 


(c) 2005 Richard Neill <backup@richardneill.org>. 


SHOMI 


This is Free Software licensed under the GNU GPL. 
==> Included in ABS Guide with script author's kind permission. 
==> (Thanks!) 


This does a backup from the host computer to a locally connected 
+ firewire HDD using rsync and ssh. 

It then rotates the backups. 

Run it via cron every night at 5am. 

This only backs up the home directory. 

If ownerships 
* then run the rsync process as root (and re-instate th c 
We save every day for 7 days, then every week for 4 weeks, 
+ then every month for 3 months. 


(other than the user's) should be preserved, 


See: http://www.mikerubel.org/computers/rsync snapshots/ 
+ for more explanation of the theory. 


E/bin/nightly-backup firewire-hdd.sh 


2S 
24 
25 
26 
27 
28 
2 
30 
Sl 
S 
33 
34 
35 
36 
Sy 
38 
9 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Syl 
52 


53 1 
54 | 
55 i 
56 | 
Sy 1 
56 d 


59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
7A 
UZ 
WS 
74 
UD 
76 
wy 
78 
[9 
80 
81 
82 
83 
84 
85 
86 
87 
88 


Know 


R) 


trite B 
LOCAL U 
MOUNT P 


SOURCE. DIR-/home/S$LOCAL USER 
E 


n bugs: 


Ideally, we want to exclude -/.tmp and the browser caches. 


If the user is sitting at the computer at 5am, 
and files are modified while the rsync is occurring, 
then the BACKUP JUSTINCASE branch gets triggered. 


TO SomboeXRODE. 


ipea) 1S d 


feature, but it also causes a "disk-space leak". 


SER-rjn 
OINT-/backup 


EGIN CONFIGURATION SI 


ECTION EREHE AE AE AE HHH TH HE HH HE HHH EH HT HH HT HH HH eH FE FE E FE EE E H 


User whose home directory should be backed up. 
Mountpoint of backup drive. 

NO trailing slash! 

This must be unique (eg using a udev symlink) 
NO tiendrai slasa = ale DOTS matter CO iesKyANe’: 


BACKUP_DEST_DIR=$MOUNT_POINT/backup/` hostname -s'.$(LOCAL USER).nightly backup 
DRY RUN-false If true, invoke rsync with -n, to do a dry run. 
Comment out or set to false for normal use. 
VERBOSE-false If true, make rsync verbose. 
Comment out or set to false otherwise. 
COMPRESS-false If true, compress. 
Good for internet, bad on LAN. 
Comment out or set to false otherwise. 
## Exit Codes ### 
E VARS NOT SET-64 
E COMMANDLINE-65 
E MOUNT_FAIL=70 
E NOSOURCEDIR-71 
E UNMOUNTED-72 
E BACKUP-73 
#### END CONFIGURATION SECTION +AA AA A H E E H H H H H H H HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE E E E HU 
Check that all the important variables have been set: 
mae qp » O OCA ems qq 
|. ez VSSouECH DERE J | | 
| e "Sn IPO Ne! ] | | 
| =z “SBACKUP DEST DIR" | 
then 
echo 'One of the variables is not set! Edit the file: $0. BACKUP FAILED.' 
exit SE_VARS_NOT_SET 
if aL 
sig [p "gu Ted ] Li Comman line paranla) 
then # Here document (ation). 
cat ««-ENDOFTEXT 
Automatic Nightly backup run from cron. 
Read the source for more details: $0 
The backup directory is S$BACKUP DEST DIR 
It will be created if necessary; initialisation is no longer required. 
WARNING: Contents of SBACKUP DEST DIR are rotated. 
Directories named 'backup.\$i' will eventually be DELETED. 
We keep backups from every day for 7 days (1-8), 
then every week for 4 weeks (9-12), 
then every month for 3 months (13-15). 
You may wish to add this to your crontab using 'crontab -e' 
# Back up files: $SOURCE DIR to S$BACKUP DEST DIR 
#+ every night at 3:15 am 


15 03 * * * /home/S$LOCAL USER/bin/nightly-backup firewire-hdd.sh 
Don't forget to verify the backups are working, 
especially if you don't read cron's mail!" 
ENDOFTEXT 
exit $E COMMANDLINE 
Fal 
# Parse the options. 


# 
qe || VSIDIRRG RUNU == eruen y CREN 

DRY RUNS NA 

echo "WARNING:" 

Selo "SEDES. JU) zx UIDyeDX Jenguspu UU 

echo "No data will actually be transferred!" 
else 


DRY RUN-"" 
fi 


if [ "SVERBOSE 
VERBOSE-"-v" 
else 
VERBOSE-"" 
i3 


"true" ]; then 


if [ "SCOMPRESS" -- "true" ]; then 
COMPRESS-"-z" 

else 
COMPRESS="" 

ipa 


# Every week (actually of 8 days) and every month, 
#+ extra backups are preserved. 


DAY_OF_MONTH=" date +%d` WP Ilse ue MONER (Oil, esie 


if [ SDAY OF MONTH = 01 J; then # First of month. 

MONTHSTART-true 
alir | SDA Or WON = 08 y 
-o SDAY_OF_MONTH = 16 \ 

-o SDAY_OF_MONTH = 24 ]; then 


# Day 8,16,24 (use 8, not 7 to better handle 31-day months) 


W 


I 


EKSTART-true 


fts 


Check that the HDD is mounted. 
At least, check that *something* is mounted here! 


+ the scsi-id by having an appropriate udev rule in 
* /etc/udev/rules.d/10-rules.local 

+ and by putting a relevant entry in /etc/fstab. 

Eg: this udev rule: 

BUS-"scsi", KERNEL-"sd*", SYSFS{vendor}="WDC WD16", 


if mount | grep $MOUNT POINT >/dev/null; then 
echo "Mount point $MOUNT POINT is indeed mounted. OK" 
else 
echo -n "Attempting to mount $MOUNT POINT..." 
» LE Hit isin ow EGG, li (tO XUOUWUWE 3E. 
sudo mount $MOUNT POINT 2>/dev/null 


if mount | grep $MOUNT POINT >/dev/null; then 


We can use something unique to the device, rather than just guessing 


SYSFS(modelj-"00JgB-00GVAO ", NAME="%k", SYMLINK-"lacie 1394d$n" 


NONNNNNNN DN NH 


Ish (E foe Se ee ft t 
@& Woy (ook «| (ex Gal dex wo IS) | 


UNMOUNT_LATER=TRUE 
echo "OK" 
# Note: Ensure that this is also unmounted 
#+ if w xit prematurely with failure. 

else 
echo "FAILED" 
echo -e "Nothing is mounted at SMOUNT_POINT. BACKUP FAILED!" 
exit SE_MOUNT_FAIL 

ifaL 

TIL 


# Check that source dir exists and is readable. 

xx ([ | =  SSOUECE DIR ] g iex 
echo "SSOURCE DIR does not exist, or cannot be read. BACKUP FAILED." 
exit $E NOSOURCEDIR 

ít 


# Check that the backup directory structure is as it should be. 
42 ILI mot, Create Lto 

# Create the subdirectories. 

# Note that backup.0 will be created as needed by rsync. 


fowr ( (Sion <1 55 y cle 


if [ ! -d SBACKUP_DEST_DIR/backup.$i ]; then 
if /bin/mkdir -p $BACKUP DEST DIR/backup.$i ; then 
d ee Qua Cy Gnd AO DAD D Croton MENTO M [le E S SE qe CRS ES AAWA 
echo "Warning: directory $BACKUP DESE DIR/backup.$i is missing," 
echo "or was not initialised. (Re-)creating it." 
else 


echo "ERROR: directory S$BACKUP DEST DIR/backup.$i" 
echo "is missing and could not be created." 
if [ "SUNMOUNT LATER" == "TRUE" ]; then 
Before we exit, unmount the mount point if necessary. 


ca 

sudo umount $MOUNT POINT && 
echo "Unmounted $MOUNT POINT again. Giving up." 
fca 


exit SE UNMOUNTED 


iE 
FIL 
done 


# Set the permission to 700 for security 
#+ on an otherwise permissive multi-user system. 
if ! /bin/chmod 700 $BACKUP_DEST_DIR ; then 
echo "ERROR: Could not set permissions on $BACKUP DEST DIR to 700." 


if [ "SUNMOUNT LATER" == "TRUE" ]; then 

# Before we exit, unmount the mount point if necessary. 
cd ; sudo umount $MOUNT POINT Y 
&& echo "Unmounted $MOUNT POINT again. Giving up." 


Haste 


exit $E_UNMOUNTED 


TES 


# Create the symlink: current -> backup.1 if required. 
# A failure here is not critical. 
cd SBACKUP  DEST DIR 
sug [p db o excuzaexeus. ]| p ilem 
sie 4 uio ilim -5 Dacko. l current p Chen 
echo "WARNING: could not create symlink current -> backup.1" 
dE ak 


BOA 3L 

222 

22:5 

224 # Now, do the rsync. 

225. exeo “Nery Coming) oaei alo 36e sov 

226 ecino “WSoumeeS chive SS(OUE(CH, Dg 

227 echo -e "Backup destination dir: S$BACKUP_DEST_DIR\n" 

228 

229 

230 /usr/bin/rsync $DRY RUN SVERBOSE -a -S delet modify-window-60 \ 
231 --link-dest-../backup.1 $SOURCE DIR S$BACKUP DEST DIR/backup.0/ 
232 

233 # Only warn, rather than exit if the rsync failed, 

234 #+ since it may only be a minor problem. 

235 4$ E.g., if one file is not readable, rsync will fail. 

236 # This shouldn't prevent the rotation. 

237 # Not using, e.g., "date +%a~ since these directories 

238 #+ are just full of links and don't consume *that much* space. 
239) 

240 a [ S2 f O WP ES 

241 BACKUP_JUSTINCASE=backup.` date +%F_%T` .justincase 

242 echo "WARNING: the rsync process did not entirely succeed." 
243 echo "Something might be wrong." 

244 echo "Saving an extra copy at: S$BACKUP JUSTINCASE" 

245 echo "WARNING: if this occurs regularly, a LOT of space will be consumed," 
246 echo "even though these are just hard-links!" 

Ay seal. 

248 

249 # Save a readme in the backup parent directory. 

250 # Save another one in the recent subdirectory. 

251 echo "Backup of $SOURCE DIR on ^hostname' was last run on \ 
252 "^date'" > SBACKUP DEST DIR/README.txt 

253 echo "This backup of $SOURCE DIR on “hostname was created on \ 


254 ^date'" > SBACKUP DEST DIR/backup.0/README.txt 


255 

256 # If we are not in a dry run, rotate the backups. 
257 [ —z "SDRY-RUN" ] && 

258 

259) Check how full the backup disk is. 


# 
260 # Warn if 90%. if 98% or more, we'll probably fail, so give up. 
261 d Noter. (olt Crain (oybur]eAbuE. ice) more than One Isliave\.)) 

262 # We test this here, rather than before 

263 #+ so that rsync may possibly have a chance. 

264 DISK FULL PERCENT-'/bin/df $BACKUP DEST DIR | 


265 ipu "UN US UV [ ewe orint S42] " | grao oE [9-9]s 

266 echo "Disk space check on backup partition \ 

267 $MOUNT POINT $DISK FULL PERCENT$ full." 

268 zie | SDS IF UI, MACON =o YO |p iem 

269 echo "Warning: Disk is greater than 90% full." 

2 UO) een 

271. stie || unu Sd FULD PERCENT -gr 9/9 lg coea 

272 Sco Vumseisgueg Diksis as; wall ANANE wis.” 

213 fa [ "SUNMOUNT LATER" == "TRUE" ]; then 

274 # Before we exit, unmount the mount point if necessary. 
BS cd; sudo umount SMOUNT_POINT && 

2NG echo "Unmounted $MOUNT POINT again. Giving up." 
2 13! iE al 

DEI exit SE UNMOUNTED 

278) iB aL 

280 

281 


282 # Create an extra backup. 
2693 Tr QUE lates Cooy tails, cane Wis. 


Zou iee ene woOBACKUP Ru US TINCAS Hass Ete hcm 
285 if l /bin/cp al $BACKUP_DEST_DIR/backup.0 \ 
286 SBACKUP_DEST_DIR/SBACKUP_JUSTINCASE 


287 then 


288 echo "ERROR: Failed to create extra copy \ 

289 SBACKUP DEST DIR/SBACKUP JUSTINCASE" 

290 aie [ "SUNMOUNT LATER" == "TRUE" ]; then 

291 # Before we exit, unmount the mount point if necessary. 

292 cd ;sudo umount $MOUNT POINT && 

299 echo "Unmounted $MOUNT POINT again. Giving up." 

294 ical 

2 9 exit $E UNMOUNTED 

296 fi 

29 ax 

298 

299 

300 # At start of month, rotate the oldest 8. 

SOUTIEN SMONIEHSIDARIN == "true"! jy then 

302 echo -e "\nStart of month. \ 

303 Removing oldest backup: $BACKUP DEST DIR/backup.15"  && 

304 /bin/rm -rf  S$BACKUP DEST DIR/backup.15 && 

305 echo "Rotating monthly,weekly backups: \ 

306 SBACKUP_DEST_DIR/backup. [8-14] => $BACKUP DEST DIR/backup.[9-15]" && 
307 /bin/mv SBACKUP DEST DIR/backup.14 $BACKUP DEST DIR/backup.15  && 


308 /bin/mv S$BACKUP. DEST DIR/backup.13 $BACKUP 
309 /bin/mv SBACKUP_DEST_DIR/backup.12 $BACKUP 


K 
K EST DIR/backup.14  && 
K 
310 /bin/mv S$BACKUP DEST DIR/backup.11 $BACKUP 
K 
K 
K 


EST DIR/backup.13  && 
EST DIR/backup.12  && 


iit de) dg) tg 


Si /bin/mv SBACKUP DEST DIR/backup.10 $BACKUP DEST DIR/backup.11  && 
312 /bin/mv SBACKUP DEST DIR/backup.9 SBACKUP DEST DIR/backup.10  && 
313) /bin/mv SBACKUP DEST DIR/backup.8 $BACKUP DEST DIR/backup.9 

314 

315 # At start of week, rotate the second-oldest 4. 

316 elif [ "SWEEKSTART" == "true" ]; then 

317 echo -e "\nStart of week. \ 

318 Removing oldest weekly backup: $BACKUP DEST DIR/backup.12"  && 

319 /bin/rm -rf  SBACKUP DEST DIR/backup.12  && 

320 

Bik echo "Rotating weekly backups: \ 

322 SBACKUP_DEST_DIR/backup. [8-11] -> $BACKUP_DEST_DIR/backup.[9-12]"  && 
323 /bin/mv $BACKUP_DEST_DIR/backup.11 SBACKUP_DEST_DIR/backup.12  && 
324 /bin/mv $BACKUP_DEST_DIR/backup.10 SBACKUP_DEST_DIR/backup.11  && 
325 /bin/mv SBACKUP DEST DIR/backup.9 SBACKUP_DEST_DIR/backup.10  && 
326 /bin/mv SBACKUP DEST DIR/backup.8 SBACKUP_DEST_DIR/backup. 9 

32,7 

328 else 

328 echo -e "\nRemoving oldest daily backup: $BACKUP DEST DIR/backup.8"  && 
330 /bin/rm -rf  SBACKUP DEST DIR/backup.8 

3S 

992. e ke 

333 

334 # Every day, rotate the newest 8. 

335 elo Ykorcarinc caly backipss y 

336 SBACKUP_DEST_DIR/backup. [1-7] -> $BACKUP DEST DIR/backup.[2-8]"  && 
S37) /bin/mv SBACKUP_DEST_DIR/backup.7 $BACKUP DEST DIR/backup.8  && 
338 /bin/mv $BACKUP DEST DIR/backup.6 $BACKUP DEST DIR/backup.7  && 
339 /bin/mv $BACKUP DEST DIR/backup.5 $BACKUP DEST DIR/backup.6  && 
340 /bin/mv $BACKUP DEST DIR/backup.4 S$BACKUP DEST DIR/backup.5  && 
SAUL /bin/mv $BACKUP DEST DIR/backup.3 $BACKUP DEST DIR/backup.4  && 
342 /bin/mv $BACKUP DEST DIR/backup.2 BACKUP DEST DIR/backup.3  && 
343 /bin/mv $BACKUP DEST DIR/backup.1 $BACKUP DEST DIR/backup.2  && 
344 /bin/mv $BACKUP DEST DIR/backup.0 $BACKUP_DEST_DIR/backup.1  && 
5345 

346 SUCCESS=true 

347 

348 

SM. sie [ "SUNMOUNT LATER" == "TRUE" ]; then 

350 # Unmount the mount point if it wasn't mounted to begin with. 


Sil cd ; sudo umount SMOUNT_POINT && echo "Unmounted SMOUNT_POINT again." 
S52. 353. 


IDS) 
354 
3595 
356 
357) 
358 
39$ 
360 
361 
3662 


nie [ VSS 
echo ' 
exito 
1E3L 


UCCESS" == "true" ]; then 
SUCCESSO 


# Should have already exited if backup worked. 
exelexo) VENOUS PALGOHD) le tats JUSt er Ciy swine Lle Thae Ciel udir) V 
exit $E BACKUP 


Example A-33. An expanded cd command 
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49 


TESTS 


TEETH 


TESTS 


eel low O) 
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EEL 
by Phil Braham 


FEFE TE FE TE FE HE FE E TE FE HE HEHE HH EEE HEH FE HE FE EE EEE EEE EERE EE 
Latest version of this script available from 
http://freshmeat.net/projects/cd/ 

FEE TE FE E AE AE FE HE TE FE HE FE E RE HE HE EH EE EH HEH ERE EE 


.cd_new 


An enhancement of the Unix cd command 


There are unlimited stack entries and special entries. The stack 
ntries keep the last cd maxhistory 

directories that have been used. The special entries can be 
assigned to commonly used directories. 


The special entries may be pre-assigned by setting the environment 
variables CDSn or by using the -u or -U command. 


The following is a suggestion for the .profile file: 


exea # Set up the cd command 
alias cd='cd_new' # Replace the cd command 
Gel =U) # Upload pre-assigned entries for 
#+ the stack and special entries 
eel =D # Set non-default mode 
alias @="cd_new Q" # Allow @ to be used to get history 


For help type: 

(xol =I (oye 

(ool ie! 
HGH HE EE E TE EE E TE FE HE EEE FE E EE FE HE E HE ERE E E EE E E EE EEE EE HEH HE HE HEE 
Version iL, 2. i 
Written by Phil Braham - Realtime Software Pty Ltd 
(realtime@mpx.com.au) 
Please send any suggestions or enhancements to the author (also at 


phil@braham.net) 


FE FEAE HE EH HT EHH HH EH HE HE HE EE EEE aE EEE 


90 4 


Bil Simus] Vea" Weg [ea] [O=O] lees) fes [sexe] [=e] \ 

$2 I-D] [ems] lesoe|9-9] Rsa] [eic 9-39] 

339 [-ee] qeS9«m-1 qeu] fou) [sg for] dmg gps wl 

54 «dir» Go to directory 

55 0-n Go to previous directory (0 is previous, 1 is last but 1 etc) 
56 n is up to max history (default is 50) 

57 @ List history and special entries 

58 @h List history entries 

59 @s List special entries 

60 -g [<dir>] Go to literal name (bypass special names) 

61 mos aber to Allow accese co Ches: Calloch Our A Vain? exe 
62 =el Change default action verbose. (S note) 

63 =D Change default action silent. (S note) 

64 -s«n» Go to the special entry <n>* 

65 -S«n» Go to the special entry «n» 

66 and replace it with the current dir* 

67 Sei jobs] (Go te) Climecieoiay Sehu 

68 and then put it on special entry <n>* 

69 Raas edirs] CoO to cimectoiy «gae 

70 and put current dir on special entry <n>* 
ql -a«n» Alternative suggested directory. See note below. 

72; eue acs |) uh INCITS (CO) KEUS 

13; -u [<file>] Update entries from <file>. 

74 If no filename supplied then default file 

TS (STEDRecka}S(23="SCDMile}) as sed 

76 F and -U are silent versions 

E Sy Print version number 

78 -dm Help 

HS SH: Detailed help 

80 

81 *The special entries (0 9) are held until log off, replaced by another 
82 entry or updated with the -u command 

83 

84 Alternative suggested directories: 

85 If a directory is not found then CD will suggest any 

86 possibilities. These are directories starting with the same letters 
87 and if any are found they are listed prefixed with -a<n> 

88 where «n» is a number. 

89 It's possible to go to the directory by entering cd -acn» 

90 on the command line. 

91 

92 The directory for -r«n» or -R«n» may be a number. 

913) For example: 

94 S elr 4. CO ico hlecory enmeiy di anc puec ie (guo. Special Gimeiay 3 
95 S cd -R3 4 Put current dir on the special entry 3 

96 and go to history entry 4 

97) $ ege =83 Go to special entry 3 

98 

oo Note that commands R,r,S and s may be used without a number 
100 and refer to 0: 
LOL $& eg =s Go to special entry 0 
102 § eg =§ Go to special entry 0 and make special 
103 entry 0 current dir 
104 S eel = d Go to history entry 1 and put it on special entry 0 
105 & gl =i Go to history entry O0 and put it on special entry 0 
106 " 
107 if S(TEST) VSCD MODE" = "PREV" 
108 then 
109 S(PRINTF) "$cd mnset" 
110 else 
ibd S(PRINTF) "$cd mset" 
EZ, 1E aL 
iw jg 
114 


TELE y DE S 


116 
3E 3,9) 
118 
19 
120 
Ad 
11212 
12:3 
124 
12/5) 
126 
12:3] 
128 
12/9; 
130 
Sal 
L332 
133) 
134 
1.35) 
SIG 
1.37 
138 
1.39 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
ISI 
152 
153 
154 
15:5; 
156 
1.57 
158 
15$ 
160 
161 
162 
16.3 
164 
165 
166 
167 
168 
169 
170 
LY 
17/2) 
IL Ws} 
174 
LHS) 
176 
1L 7/7) 
178 
LHS, 
180 
181 


{ 


COl lou 
S(PRINTF) "Ss" " 


a 


e 
S 
È 
a 


he previous directories (0-$cd maxhistory) are stored in the 
nvironment variables CD[0] - CD[S$cd maxhistory] 

imilarly the special directories S0 - $cd maxspecial are in 
he environment variable CDS[0] - CDS[S$cd maxspecial] 

nd may be accessed from the command line 

he default pathname for the -f and -u commands is $CDPath 

he default filename for the -f and -u commands is $CDFile 

et the following environment variables: 


CDL PROMPTLEN .- Set to the length of prompt you require. 
Prompt string is set to the right characters of the 
current directory. 

If not set then prompt is left unchanged 

CDL PROMPT PRE - Set to the string to prefix the prompt. 
Default is: 


acn- roots — WOWWrWwyeltoLlssS4gNWISNU (sees colour to bluss 
TOOL: VAAN lOl Sim \\INY leete colour to recs 
CDEBE ROME MEO Set = SEI dee Che Serine Ter Wiese elas NE . 


Default is: 
non-root: \"\\[\\e[00m\\]$\" 
(resets colour and displays $). 
root: NNN[NNe[00mNN] #\" 
(resets colour and displays #). 
CDPath - Set the default path for the -f & -u options. 
Default is home directory 
CDFil Set the default filename for the -f & -u options. 
Default is cdfile 


cd version 


cd version 


{ 


} 


0 


printf "Version: $(VERSION MAJOR).$(VERSION MINOR) Date: $(VERSION DAT 


Truncate right. 


params: 


jal. = Sieieninie) 
pA — IMG EO iETewue enki. EO 


SICA. Se cae slimy cel 


CCL makene e ewa (0) 


{ 


local 
Local 
local 
local 
local 
se xl 
then 


tlen=${2} 
plen=$ {#1} 
str-"$(1)" 
Gla sexe 

ELl lers g=- 


TEST) S$(plen) -1le ${tlen} 


COWS {is itera) 


else 


let diff=${plen}-S${tlen} 
elen=3 

ux SAS) S Cher) le 2 
then 


E}\n" 


NONNNNNN NH 
(S9). =] (x) Gr) dE Ge Jm» [— 


N N 
N H 
c» Wo) 


Se oce che cR co dk 


let elen=${diff} 
iP dL 
tlen=-${tlen} 
let tlen=S{tlen}+S${elen} 
tees i Filler (0) e eese] S eee wie Leia} 


it3L 


Three versions of do history: 


cd dohistory = packs history and specials side by side 
cd dohistoryH - Shows only hstory 
cd dohistoryS - Shows only specials 


Gul Colaisicomy (0) 


{ 


cd getrc 
S{PRINTF} "History: Mn" 


local —3 count=S{cd_histcount } 
wails Spur Steowre}! -06 0 
do 
cel rice crne: “SCD (Cowme |}! Sieci tenar} 
SRO IE VeZo a Stes lehar e S ee lelara V Steouiate | "Sel wy 
Ciel rice rume "USICIDIS |lComiae |} SiCel_ renens) 
S{PRINTF} "S$d %-S{cd_rchar}.${cd_rchar}s\n" $(count) "S{tcd}" 
count=$ {count }-1 
done 


cd_dohistoryH () 


{ 


cd getrc 

S{PRINTF} "History: Mn" 

local -i count-$(cd maxhistory) 

wile ie SdeowwnE -G8 0 

do 
S{PRINTF} "S{count} $-$(cd flchar).$(cd flchar)s Mn" 
count=$ {count }-1 

done 


ecl_Colaisicorys () 


{ 


{ 


cd getrc 
S{PRINTF} "Specials:\n" 
local -i count=${cd_maxspecial} 
waila Sane fSdeugwuns -08 0 
do 


S(PRINTF) "SS{count} %-${cd_flchar}.${cd_flchar}s\n" 


count=$ {count }-1 
done 


Ciel ersieicre. 10) 
Cel seilelieie=S (uw =e | cus = X5 
rows i prine S2 S93 Jj" | awk =e YY prine SA V) 
suit SMHS} Sec lenene ime: (0) 
then 


ed_lchar=S {ed_flehar}/2—-5 
cd rchar-$(cd flchar)/2-5 
cd flchar-$í(cd flchar)-5 
else 
cd_flchar=$ {FLCHAR:=75} 
# cd_flchar is used for for the @s & @h history 


${CD[S$count] } 


${CDS[$count] } 


248 cd_lchar=$ {LCHAR:=35} 

249 cd_rchar=$ {RCHAR:=35} 

250 ie ab 

251L j 

252 

253 œ coselecc iom (0) 

254 { 

255 local -i nm=0 

296 cd doflag-"TRUE" 

259 ae SAS VSD MODI YS WIETEdmg/AU 

2/59) then 

259) ade Suet Seele" 

260 then 

261 cd_npwd=0 

262 dE aL 

263 igi. 

264 tm=S$ (echo "S{cd_npwd}" | cut -b 1) 

265 ae S EST SIS torm a e eS 

266 then 

267 pm=S (echo "S${cd_npwd}" | cut -b 2) 

268 mS (relay Yles moweh Y || cre el Som 12) 

269 Case WS (joj! aim 

270 a) cd_npwd=${cd_sugg[$nm]} ;; 

DUAL s) cd_npwd="S{CDS[Snm]}" ;; 

272 S) ecd npwd-"$(CDS[$nm])" ; CDS[$nm]= pwd ;; 

BUS r) cd npwd-"$2" (lol yexexeuDabweeuown F Cel_closelectaom VSI WgUprs 
274 R) (erel-oeyexwoE- S2 m CDS Sam= owe p rclicleiseulacicateig WoW Ug» 
2 35 esac 

2 165 iE 3b 

VAT 

278 xr igfe sur VS teclinjouycl ey I= WU Sa US mex ^ X 

27S) t= WuU..w m UST qme" he Siteclimaxiulsicoiey | Se/clew awl 25i 
280 then 

281 cd_npwd=$ {CD [$cd_npwd] } 

Zo, else 

23S) case "Scd_npwd" in 

284 @ cd dohistory ; cd_doflag="FALSE" ;; 

2/95) Qh) cd dohistoryH ; cd doflag-"FALSE" ;; 

286 (i59). eel cloinisicormys p ecl clotlage VP AnSia n5 

287 —19) gol lm pf «yel olo lees gnus 2 5 

288 =R) «eol ien 2 Cel oll AUS ps 

289 =i) «eL ewe WISIN S2 9 CO eoe aee Mais no 
29D =i) eel itioil@evel "SiO SA p «el elo. Sinisa n5 
29 =E else. "uNOSEON" S2 5 «rel gloilc— Pubs p 
292 U) Ce voloc! VNOSEOWY SA EE Im Od tac S HAT CBE, 
293 =C) (ego s2w pp 

294 -9D) «exl «ewe Ie ecel eer lea IMAL Sig Eo 

295 =D) Ce Chicken Of Ce eei laiem TAS ep 

296 -r) cd npwd-"$2" CCl roxexelbataics(0). 6 Texel Closicilecicaioin SIRAS 
297 =2) «exl sequo S27 pn CDSIO]- gel p eclcloseileceiem VSI USAa p 
298 -S) cd npwd-"$(CDS[0]J" ;; 

299 -S) cd_npwd="${CDS[0]}" ; CDS[0]- pwd' ;; 

300 -V) cd version ; cd doflag-"FALSE";; 

301 esac 

302 iral 

Ss F 

304 

505 cce endem (0) 

306 ( 

307 if S(TEST) "S(CD MODE)" = "PREV" 

308 then 

309 CD. MODE-"" 

310 s SAS} Sil ep di 

511. 1i then 

312 S{PRINTF} "S${cd_mset}" 

313 1E 3L 


314 
34.8 
316 
317 
318 
EXE 
320 
S2 
322 
323 
324 
3215 
326 
327 
328 
329 
330 
33 
332 
333 
334 
335) 
336 
337 
338 
389 
340 
341 
342 
343 
344 
345 
346 
347 
348 
349 
350 
35 
552 
393 
354 
395 
356 
357 
358 
259 
360 
361 
362 
363 
364 
365 
366 
367 
368 
369 
370 
3i 
372 
S73) 
374 
S79 
376 
377 
378 
379, 


else 
CD MODE-"PREV" 
stie Saou} Sil Sey il 
then 
S{PRINTF} "$ 
if aL 
I3 


cd fsave () 


{ 


{cd_mnset}" 


local sfile=${CDPath}${2:-"SCDFile"} 


ati Sf ASI 
then 


} WSW 


= "SHOW" 


S(PRINTF) "Saved to 


iE aL 


S{RM} -f ${sfile} 


local -i count=0 


echo "CD[$count]=\"${CD[$count]}\"" 


count) -le $ 


count }+1 


count) -le $ 


$s\n" $sfile 


(cd maxhistory) 


(cd maxspecial) 


echo "CDS[$count]-N"S(CbS[$count])N"" 


while ${TEST} $ 
do 
count-$ 
done 
count=0 
while ${TEST} $ 
do 
Coume=S 
done 
} 
Ccmuploadan() 


{ 


count }+1 


local sfile=${CDPath}${2:-"SCDFile"} 
} NUS OMIT = "SHOW" 


sit SP WAS 
then 


S{PRINTF} "Loading f 


Ea 
${sfile 


cd new () 


{ 


} 


local =i Corme 
local -i choose=0 


cd_npwd="${1}" 
cd specDir--1 
eel closes UST EU. Wwedpe su 


ie SMS) iere okesleg = w 


then 


sap ME 


then 


3E 3L 
command cd "$í(cd npwd)" 2>/dev/null 
sue Si was} $2 ee i 


then 


${PRINTF} 


aS SHUT 


rom s\n" S${sfile} 


count-$cd ma 
while ${TES 
do 


RUTEM 


wots ""pwd^ " 


xhistory 
} $count -gt 0 


>> S(sfile) 


>> S(sfile) 


CD[$count]-$(CD[S$count-1]) 


coun 
done 
CD[0]=" pwd” 


"Unknown dir: 


t=${count }-1 


s\n" "S${cd_npwd}" 


local -i ftflag=0 
roe aL zum "Ecol suyo) ws 


do 
aude Spe x WS dari! 
then 
SHE SAWS} S eeil exp 0 
then 
S{PRINTF} "Suggest:\n" 
ftflag-1 
acak 
S{PRINTF} "Nt-a$(choose) %s\n" "Si" 
cd_sugg[$choose]="${i}" 
choose=${choose}+1 
i3. 
done 
3 3L 
if aL 


ie S(WASw} Secl spescDiiz) =me il 


then 
CDS [${cd_specDir}]=* pwd 
ea 
if S(TEST) ! -z "S${CDL_PROMPTLEN}" 
then 
cd right trunc "S$(PWD)" ${CDL_PROMPTLEN} 


cd rp-$(CDL PROMPT PRE)S$(tcd)$(CDL PROMPT POST) 
export PS1="$(echo -ne ${cd_rp})" 


Haste 


FE HE AE HH AE FE a HE HEE HE HEE EE EEE aE EEE EEE 


# # 
# Initialisation here # 
# # 


FE FEAE EH HE HE HEE EEE HE EE EEE EE EEE EEE 
# 

VERS ION_MAJOR="1" 

VERSION MINOR-"2.1" 

VERSION DATE-"24-MAY-2003" 


alias cd-cd new 


Set up commands 


RM=/bin/rm 
TEST=test 
PRINTF-printf # Use builtin printf 
HEHEHE RE EE EEE EE EEE HE EE EEE EE EH HE EE EEE EE EEE HE EE EEE EE EEE HE EE EEE EE HH HE EE HEH 
# 
Change this to modify the default pre- and post prompt strings. # 
These only come into effect if CDL_PROMPTLEN is set. # 
# 
HERE HHH EE EEE EE EEE HE EE EEE EE EH HE EE EE EE EEE HE EE EEE EE ERE HE EEE EE HE EE HE HEH 


ae SMSL SMA} ee, O 


then 
CDL PROMPT PRE-$(CDL PROMPT PRE:-"SHOSTNAMEG") 
CDL PROMPT PRE-S$(CDL PROMPT PRE:-"NN[NNe[01;31mNN]") 4 Root is in red 
CDL PROMPT POST-$[(CDL PROMPT POST:-"NNV[NNe[00mNN] £") 
else 
CDL PROMPT PRE-S$(CDL PROMPT PRE:-"NN[NNe[01;34mNN]") 4 Users in blue 


CDL_PROMPT_POST=$ {CDL_PROMPT_POST:="\\[\\e[00m\\]$"} 
ig aL 
TEGERE ERE TE FE FE HE HE EEE FE HE EE FE FE HE ERE EE HE E TE FE EE ERE EE EE EE E EE E E EE E E ERE 
# 
# cd_maxhistory defines the max number of history entries allowed. 


typeset -i cd maxhistory-50 


HEHE HHT HHH EH HT FE AE FE AE FE Ho EE EEE aE EE EEE 


cd_maxspecial defines the number of special entries. 
typeset -i cd_maxspecial=9 


FE FEAE EH HE AE FE FE AE FE HE E FE AE FE FE FE aE EE EE EE aE EEE 


cd_histcount defines the number of entries displayed in 
* the history command. 
typeset -i cd histcount-9 


HPT HE EEE RE EEE HE HE EEE EE EE HE EE EEE ERE EE ER HE EE HEH RE EE 
export CDPath=S {HOME} / 

Change these to use a different 

+ default path and filename 

export CDFile=${CDFILE:=cdfile} # for the -u and -f commands 


rE HHH EH HT HEE HE EE EEE EEE aE FE E 


typeset -i CCl kehee (rel ehen Col it lenena 


# This is the number of chars to allow for the 
cd flchar-S$(FLCHAR:-75) 4-4 cd flchar is used for for the 8s & @h history 
typeset -ax CD CDS 
# 
cd_mset="\n\tDefault mode is now set - entering cd with no parameters \ 


has the default action\n\tUse cd -d or -D for cd to go to \ 
previous directory with no parameters Vn" 
cd_mnset="\n\tNon-default mode is now set - entering cd with no \ 
parameters is the same as entering cd 0\n\tUse cd -d or A 

-D to change default cd action\n" 


# # 


<<DOCUMENTATION 


Written by Phil Braham. Realtime Software Pty Ltd. 
Released under GNU license. Fr to use. Please pass any modifications 
or comments to the author Phil Braham: 


realtime@mpx.com.au 


cdll is a replacement for cd and incorporates similar functionality to 
the bash pushd and popd commands but is independent of them. 


This version of cdll has been tested on Linux using Bash. It will work 
on most Linux versions but will probably not work on other shells without 
modification. 


Introduction 


cdll allows easy moving about between directories. When changing to a new 
directory the current one is automatically put onto a stack. By default 
50 entries are kept, but this is configurable. Special directories can be 
kept for easy access - by default up to 10, but this is configurable. The 
most recent stack entries and the special entries can be easily viewed. 


The directory stack and special entries can be saved to, and loaded from, 
a file. This allows them to be set up on login, saved before logging out 
or changed when moving project to project. 


In addition, cdll provides a flexible command prompt facility that allows, 
for example, a directory name in colour that is truncated from the left 


if it gets too long. 


SSE Line) va Ceu 


Copy cdll to either your local home directory or a central directory 
such as /usr/bin (this will require root access). 


Copy the file cdfile to your home directory. It will require read and 
write access. This a default file that contains a directory stack and 


special entries. 


To replace the cd command you must add commands to your login script. 
The login script is one or more of: 


/etc/profile 
~/.bash_profile 
~/.bash_login 
~/.profile 

~/.bashre 
/etc/bash.bashrc.local 


To setup your login, -/.bashrc is recommended, for global (and root) setup 
add the commands to /etc/bash.bashrc.local 


To set up on login, add the command: 


«dir»/cdll 


For example if cdll is in your local home directory: 


^/cdll 
Iie Sum Ause Aon itle 
/usr/bin/cdll 


If you want to use this instead of the buitin cd command then add: 


alias cd='cd_new' 


We would also recommend the following commands: 


alias Q-'cd new @' 
(ol =U) 
(ol =D 


If you want to use cdll's prompt facilty then add the following: 


CDL PROMPTLEN-nn 


Where nn is a number described below. Initially 99 would be suitable 


number. 


Thus the script looks something like this: 


HHEPHHEEEEEEEEEEEEHE E HEHE 
# CD Setup 
HHEPFHEEEEHEEEEEEEEE EEE HEHE 
CDL_PROMPTLEN=21 
/usr/bin/cdll 

alias cd='cd_new' 

alias Q8-'cd new @' 

(eio =U 

evel ID) 
HHEPHHEEEEHEEEEEEEEHE EEE HE EEE 


# 


# 


# 


Hr HH HH EH HEH HE EE EE E FE AE FE F 


FEFE EE HE TE EEE E TE EE HH EE EEE EE EE ERE HE ERE E E E E E HE EEE 
Allow a prompt length of up to 21 characters 
JGesbiEsLenlabexe vex 

Replace the built in cd command 

Allow @ at the prompt to display history 
Upload directories 

Set default action to non-posix 

TERRE EE ERE ERE EE RE EE ERE EE EEE 


The full meaning of these commands will become clear later. 


There are a couple of caveats. 
without calling cdll, then the directory won't be put on the stack and 
also if the prompt facility is used then this will not be updated. Two 


If another program changes the directory 


(yy Ln), (OX. on) fe» (ex. «9» 
(g9) «| (ox) Gr) dmm GS) IS 


a OY 
ix» f H 
ex Wo 


programs that can do this are pushd and popd. To update the prompt and 


stack simply enter: 


ea 


Note that if the previous entry on the stack is the current directory 


then the stack is not updated. 


Usage 
ed [dir] [0-9] [@[s|h] [-g <dir>] [-—d] [=D] [=r<n>] 

[Glaize ||O=S)]) [esse] [<cliz>|O=—9] [=s<a>] Ssa] 
[-w] [9] [es] liem] [m] [ae law 
«dir» Go to directory 
()) im Goto previous directory (0 is previous, 

J| xe lasg lowe di ieee) 

ig 3ug tio To mecs imLedEguew (cleitenillic is 50) 
@ List history and special entries (Usually available as $ @) 
en List history entries 
Gs List special entries 
-g [<dir>] Go to literal name (bypass special names) 

mares ale} to allow access tO Circes edle! V. 17. Vm" xe 
=o Change default action verbose. (S note) 
=D) Change default action silent. (S note) 
-s«n» Go to the special entry «n» 
Ec Go to the special entry «n» 

and replace it with the current dir 
=ie<in=  |i<chire> || (Go 19 elüeectonw «eBue 
and then put it on special entry «n» 
Stine Leca (ere ice (isecxetouew «etse. 
and put current dir on special entry «n» 

-a«n» Alternative suggested directory. See note below. 
=I [pesti] iil ntries to <file>. 


-u [<file>] Update entries from <file>. 
If no filename supplied then default fil 
F and -U are silent versions 


=y Print version number 

-—igi Help 

=lal Detailed help 
Examples 


These examples assume non-default mode is set (that is, 
parameters will go to the most recent stack directory), 


have been set up for cd and @ as described above and that cd's prompt 


facility is active and the prompt length is 21 character 


/home/phil$ @ 

List the entries with the @ 
alaL SHE ose 8 

Output of the @ command 


Skipped thes ntries for brevity 


1 /home/phil/ummdev S1 /home/phil/perl 
Most recent two history entries 
0 /home/phil/perl/eg SO /home/phil/umm/ 


and two special entries are shown 


/home/phil$ cd /home/phil/utils/Cdll 
Now change directories 
/home/phil/utils/Cdll$ @ 

Prompt reflects the directory. 


a (eh 1) 


cd with no 
that aliases 


S. 


ummdev 


is used 


644 
645 
646 
647 
648 
649 
650 
GS 
652 
G53 
654 
695 
656 
657 
658 
659 
660 
661 
662 
663 
664 
665 
666 
667 
668 
669 
670 
671 
672 
673 
674 
ETS 
676 
677 
678 
(9:79; 
680 
681 
682 
683 
684 
685 
686 
687 
688 
689 
690 
Gom 
692 
623 
694 
629 
696 
697 
698 
699 
700 
Tg) 
702 
103 
704 
705 
706 
707 
708 
709 


To 


To 


TO 


To 


LO) 


IL ree S 
# New history 


1 /home/phil/perl/eg S1 /home/phil/perl 

# History entry 0 has moved to 1 

0 /home/phil SO /home/phil/umm/ummdev 
# and the most recent has entered 


go to a history entry: 


/home/ ist 1L/weestibe/ CUIUS «eol i 

# Go to history entry 1. 
/home/phil/perl/eg$ 

# Current directory is now what was 1 


go to a special entry: 


/home/phil/perl/eg$ cd -s1 
# Go to special entry 1 
/home/phil/umm/ummdev$ 
# Current directory is S1 


go to a directory called, for example, 1: 
Pisoni) olan ILS eel =e; 1 


# -g ignores the special meaning of 1 
/home/phil/1$ 


put current directory on the special list as S1: 


€. =i . # OR 
Gel —EUL 5 # These have the sam ffect if the directory is 
#+ . (the current directory) 


go to a directory and add it as a special 
The directory for -r«n» or -R«n» may be a number. 
For example: 
S eel -r3 4 Co to hlerory (uus 4! uel pur le Om SPECA emciy 3 
$ eel SRS 4 purt Curran: Clie qox the Special enciy 3 and cio Te 
history entry 4 
$ eol -93 Go to special entry 3 


Note that commands R,r,S and s may be used without a number and 
refer to 0: 
$ eg = Go to special entry 0 
§ gel =§ Go to special entry 0 and make special entry 0 
current dir 
§ eel sie i Go to history entry 1 and put it on special entry 0 
S (gl =e Go to history entry O0 and put it on special entry 0 


Alternative suggested directories: 


If a directory is not found, then CD will suggest any 
possibilities. These are directories starting with the same letters 
and if any are found they are listed prefixed with -a<n> 

where «n» is a number. It's possible to go to the directory 

by entering cd -a«n» on the command line. 


Use cd -d or -D to change default cd action. cd -H will show 
current action. 


The history entries (0-n) are stored in the environment variables 


CDON = CDa 
Similarly the special directories S0 9) eee: alin Ela nvironment 
variable CDS[0] - CDS[9] 


and may be accessed from the command line, for exampl 


mdp cmd] xp cep cp md] mb c cm] 
a «| (x) Gr dE GS) Jj «m» 


Jus =i S4 DS [s] 
car SCD) [ES] I ELEn TAE 


The default pathname for the -f and -u commands is ~ 
The default filename for the -f and -u commands is cdfile 


Configuration 
719 ============= 
720 
TAT The following environment variables can be set: 
712222 
12:3) CDL PROMPTLEN .- Set to the length of prompt you require. 
724 Prompt string is set to the right characters of the current 
T25 directory. If not set, then prompt is left unchanged. Note 
WAG that this is the number of characters that the directory is 
VAY shortened to, not the total characters in the prompt. 
728 
729 GDE PROMPT PRE — Set to the string to prefix the prompt. 
730 Default is: 
TESNIL mom=roors VIS summ] V (sens colour tee oia) 
732 TOORE 8 CN [ES se Tg S Tan 1] © (Gers. colour To Tac) a 
733 
734 CDL PROMPT POST = SE co Che STAG ito SEI Clas AEAN o 
735) Default is: 
736 non-root: "\\[\\e[00m\\]$" 
JUST) (resets colour and displays $). 
738 root: "NN[NNe[00mNN] 4" 
TSS (resets colour and displays #). 
740 
741 Note: 
742 CIDG EROM 1 RING 4. POST (unl 15 
743 
744 CDPath - Set the default path for the -f & -u options. 
745 Default is home directory 
746 CDE Set the default filename for the -f & -u options. 
747] Default is cdfile 
748 
749 
750 There are three variables defined in the file cdll which control the 
TS number of entries stored or displayed. They are in the section labeled 
752 'Initialisation here' towards the end of the file. 
758 
754 cd_maxhistory - The number of history entries stored. 
7/55) Default is 50. 
756 cd maxspecial - The number of special entries allowed. 
TSF Default is 9. 
TSS cd_histcount — The number of history and special entries 
TSS displayed. Default is 9. 
760 
761 Note that cd_maxspecial should be >= cd_histcount to avoid displaying 
762 special entries that can't be set. 
763 
764 


765 Version: 1.2.1 Date: 24-MAY-2003 


766 
767 DOCUMENTATION 


Example A-34. A soundcard setup script 


1 #!/bin/bash 


Mop p opp ppppÀ: 
O io 0» -1 O O1 i& (0. M P). O (o 0 -1 O O1 4 C ID 


NNN PN 
ge (eX du» f 


25 


NNN 
(so) 1] 19») 


C9 CO NO 
j— e» No) 


C0 CO CO CO CO CO CO CO 
Wer 9 SS) (ex. Gal dev fe) du» 


A e 
i €» 


PoP PB am 
Ow WN 


46 


od 
-1 


48 


Ov Ov OF e 
oy [LS €» Wwe 


(On Cal (Onb Gar Kon) Gm} Oni 
Wer toe) c3 (e»y (On SS e» 


ON OW OYSEOYSONEOWEOY 
Qy (nb dex (63) [n9 [— «e» 


oO) 
-1 


Soundcard-on.sh 


SEUSS etnos 
http://www.thinkwiki.org/wiki 
/Script for configuring the CS4239 sound chip in PnP mode 
ABS Guide author made minor changes and added comments. 
Couldn't contact script author to ask for permission to use, but 
* the script was released under the FDL, 
+ so its use here should be both legal and ethical. 


Sound-via-pnp-script for Thinkpad 600E 
+ and possibly other computers with onboard CS4239/CS4610 
+ that do not work with the PCI driver 

ae enel hue 


Mkarcher 


not recognized by the PnP code of snd-cs4236. 


Also for some 770-series Thinkpads, such as the 770x. 
Bun aS ToC SE 


of course. 


These are old and very obsolete laptop computers, 
+ but this particular script is very instructive, 
+ as it shows how to set up and hack device files. 


# Search for sound card pnp device: 


for dev in /sys/bus/pnp/devices/* 


do 


grep CSC0100 $dev/id > /dev/null && WSSDEV-$dev 
grep CSCO110 $dev/id > /dev/null && CTLDEV-$dev 


done 


WSSDI 


Ou 7 


CTLDI 


(s 
MO cm 


/sys/bus/pnp/devices/00:07 
/sys/bus/pnp/devices/00:06 


Activate devices: 
Thinkpad boots with devices disabled unless "fast boot" is turned off 
+ (in BIOS). 


These are symbolic links to /sys/devices/pnp0/ 


echo activate > $WSSDEV/resources 
echo activate » SCTLDEV/resources 


# Parse resource settings. 


{ read 
read 
read 
read 
read 
read 
read 

The 


afe [[ 


# Discard 


bla 
bla 
bla 
bla 
bla 
bla 


"bla's" 


Hack: 
zr (Que POTON 


POr wii 
port2 
port3 
aliter 

dmal 
dma2 


sb 


"State = active" (s below). 


are labels ain the frest field: Waly, M Wietecheci, M (exo 
These are discarded. 


Wield IE:WEJEDIO/SSS ORES ere: joOrels WSS, Worries 
(unneeded) 


WALICin INCI ILSIPIMeE PONES chess. jOoricils OM, joomezs slo, jxowiess WISIS 


A^AA^A^^ 


(ACPI bios seems to be wrong here, the PnP-card-code in snd-cs4236.c 
+ uses the PnPBIOS port order) 

Detect port order using the fixed OPL port as reference. 
$(port2$2-*) 


= 0x388 ] 

Strip out everything following hyphen in port address. 
SO, iit porel sue 9x530-05537 

we're left with 0x530 -- the start address of the port. 


68 then 


69 # PnPBIOS: usual order 

70 port=$ {port1%%—*} 

pai oplport-$(port2$$-*) 

72 else 

ws # ACPI: mixed-up order 

74 port=S$ {port3%%-* } 

75 oplport-$í(portl$$-*) 

VE 3x 

77 } < SWSSDEV/resources 

78 To see what's going on here: 
p 

80 cat /sys/devices/pnp0/00:07/resources 
81 

82 state = active 

83 1o 02590-02537 

84 io 0x388-0x38b 

85 suo (0px22)0).— (05523) 9 

86 TAES, 

87 dma 1 

88 dma 0 

89 free walat JbaexeLs im. irst risio (ebuexeiaseelexl) « 
90 

91 


92 ( read # Discard first line, as above. 
93 read bla portl 
94 cport=S {port1%%—* } 
95) # ES 
96 # Just want _start_ address of port. 
97 } « SCTLDEV/resources 
98 
99 
100 4 Load the module: 
101 
102 modprobe --ignore-install snd-cs4236 port-$port cport-$cportN 
103 fm port-S$oplport irq-$irq dmal-$dmal dma2=Sdma2 isapnp-0 index=0 
104 4 See the modprobe manpage. 
105 
106 exit $? 


Example A-35. Locating split paragraphs in a text file 


1 #!/bin/bash 
2 # find-splitpara.sh 
3 # Finds split paragraphs in a text file, 
4 #+ and tags the line numbers. 
E 
6 
7 ARGCOUNT-1 4 Expect one arg. 
8 E WRONGARGS-65 
9 
10 file="S1" # Target filename. 
11 lineno-1 # Line number. Start at 1. 
12 Flag=0 # Blank line flag. 
13 
14 if [ $4 -ne "SARGCOUNT" ] 
15 then 
16 echo "Usage: "basename $0?! FILENAME" 
17 exit $E WRONGARGS 
Jg SE ab 
1¢ 
20 file_read () # Scan file for pattern, then print line. 


FANE 
22 while read line 
25. ko 


25 mie || ES SA =~ a du Se ac db gl 

26 then # Line begins with lc character, following blank line. 
2] echo -n "$lineno:: " 

28 echo "ling" 

ZY) 1E3L 


32 aie PE AEST AE uh "Ec US 

33 then # If blank line, 
34 Flag=1 #+ set flag. 

35 else 

96 Flag=0 

37 3E 3L 


39 ((lineno++) ) 


41 done 
42 ]j « SsesLi jr INechiaecte LLS sbowe(o) ENCE TONT Is. SEEEN 


44 file read 


47 exit $? 


50 # 
51 This is line one of an example paragraph, bla, bla, bla. 
52 This is line two, and line three should follow on next line, but 


54 there is a blank line separating the two parts of the paragraph. 
Sis 


57 Running this script on a file containing the above paragraph 
58 yields: 


0) “ae there is a blank line separating the two parts of the paragraph. 


63 There will be additional output for all the other split paragraphs 
64 in the target file. 


Example A-36. Insertion sort 


1 #!/bin/bash 
2 insertion-sort.bash: Insertion sort implementation in Bash 
3 Heavy use of Bash array features: 
4 d (string) slicing, merging, etc 
5 URL: http://www.lugmen.org.ar/-jjo/jjotip/insertion-sort.bash.d 
6 #+ /insertion-sort.bash.sh 
7 
8 Author: JuanJo Ciarlante <jjo@irrigacion.gov.ar> 
9 Lightly reformatted by ABS Guide author. 
10 License: GPLv2 
dd Used in ABS Guide with author's permission (thanks!). 
12 
LB Test with: al imgertion- -SOTT oasa E 
14 Ors bash insertion-sort.bash E 
LS The following *doesn't* work: 


16 sh insertion-sort.bash -t 


17 Why not? Hint: which Bash-specific features are disabled 

18 #+ when running a script by 'sh script.sh'? 

19 

20 : S(DEBUG:-0) # Debug, override with:  DEBUG-1 ./scriptname 

2 Parameter substitution -- set DEBUG to 0 if not previously set. 
22 


23 Gloloall. aiwenys Walasic! 
24 typeset -a list 


25 Load whitespace-separated numbers from stdin. 

Ze alae (p Wasa Wee qe season 

27 DEBUG=1 

28 read -a list « <( od -Ad -w24 -t u2 /dev/urandom ) # Random list. 
29 d ^ ^ process substition 

30 else 

SHE read -a list 

EE 

33 numelem=${#list[*]} 

34 


35 4 Shows the list, marking the element whose index is $1 
36 #+ by surrounding it with the two chars passed as $2. 

37 # Whole line prefixed with $3. 

38 showlist () 


39 { 

40 exelao) HSSISH isisie EI eOsSily Sy 2eOe iL $94135w-[ S92] HsS4293121j3 Silas [ej] sS: 
41 } 

42 

43 # Loop _pivot_ -- from second element to end of list. 

44 for(( i=1; i<numelem; i++ )) do 

45 ( (DEBUG) )&&showlist i "[]" " " 

46 # From current | pivot , back to first element. 

47] tore (( sexo af 3—— )) clo 

48 # Search for the 1st elem. less than current "pivot" 
49 E Stirs [jal =le wSquseu[x])" Jp] tee orsak 

50 done 

Bil (( i==j )) && continue ## No insertion was needed for this element. 
52 if o o c Nove ial] (quw) 1o tine lex oi reelde 

5.3 Lisie= (Sills [H8 9108 si Sills fay} Spbusse (si) }\ 

54 d 19/77 aaa}; (i) {3} 

55 Ss eee aS S [ey] gata by 

56 # i 3nedbo Lal} fiale last) 

57 ((DEBUG))&&showlist j "<>" "x" 

58 done 

59 

60 

61 echo 

o2 Techo o cec M 

63 echo $'Result:\n'S${list[@] } 

64 

6S Git G2 


Example A-37. Standard Deviation 


!/bin/bash 
sd.sh: Standard Deviation 


The Standard Deviation indicates how consistent a set of data is. 
It shows to what extent the individual data points deviate from the 
+ arithmetic mean, i.e., how much they "bounce around" (or cluster). 
It is essentially the average deviation-distance of the 
+ data points from the mean. 


Wo) (eer c oy) Gal dE ($5 [SOP [S 


NPRPRPPRP PPP PY 
O 00-100 50N^OÓO 


OEC NOISE ESSE SS Eb ME SO SENOS M ESO MES 
j— Ee Wel (e Sa] ix, (nb dev Go de» [5 


C0 CO CO CO CO CO CO CO 
Wey (Gs SS) cx Gal dev (e9 du» 


Bop 
[- €» 


C1 Or. fb Sb PP am au 
i (3p Wer (oe) 3] (ex, (On! Wes. (eor [n5 


[91 
N 


PITIAS GE (On) (m) Oni 
We toe) cJ (exy Gn) des eS) 


60 


=| Sx) Ten cep) fon) ery fen Gay x) ON OY 
I= «m.1We) (ex SS) Cx; (nb dev Eo) des [5 


ay — xa 
Owe WN 


To calculate the Standard Deviation: 


1 Find the arithmetic mean (average) of all the data points. 
2 Subtract each data point from the arithmetic mean, 
and square that difference. 
3 Add all of the individual difference-squares in 4 2. 
4 Divide the sum in # 3 by the number of data points. 
This is known as the "variance." 


5 The square root of # 4 gives the Standard Deviation. 
# 
count=0 # Number of data points; global. 
SC-9 # Scale to be used by bc. Nine decimal places. 
E DATAFILE-90 # Data file error. 
Set data Emi 
if [ ! -z "S1" ] # Specify filename as cmd-line arg? 
then 
datafile-"$1" # ASCII text file, 
else #+ one (numerical) data point per line! 
datafile-sample.dat 
agak # See example data file, below. 
ace [p db eS VECE Le j 
then 
echo "\""Sdatafile"\" does not exist!" 
exit $E DATAFILE 
iE aL 
# 
arith_mean () 
{ 
local rt=0 # Running total. 
local am=0 # Arithmetic mean. 
local ct=0 # Number of data points. 
while read valu # Read one data point at a time. 
do 
rt=S (echo "scale-$SC; Srt + $value" | bc) 
( «E ») 
done 
sue (Scine Weieelle=SSCe SE 7 Sec! || Tx) 
Scho Sane Tertr SCE # This function "returns" TWO values! 
s Cawiesome Wate lal tiene wali mot work nit Sete > 259551 
# To handle a larger number of data points, 
#+ simply comment out the "return Sct" above. 
p «Solet sc la” r rees ism data Pile, 
eel X) 
{ 
meanli-$1 4 Arithmetic mean (passed to function). 
n-$2 # How many data points. 
sum2=0 # Sum of squared differences ("variance"). 
avg2=0 # Average of $sum2. 
sdev=0 # Standard Deviation. 
while read valu # Read one line at a time. 
do 
diff=$ (echo "scale-$SC; $meanl - $value" | bc) 
# Difference between arith. mean and data point. 
ChiicZ=S (eel. WaCeello=Ssiee Ses = SCHEEN || Jexey) Gj; fSXenbteuctsiels 
sum2=S (echo "scale-$SC; Ssum2 + Sdif2" | bc) # Sum of squares. 


76 done 

1:7 

78 avg2-$ (echo "scale-$SC; $sum2 / $n" | bc) # Avg. of sum of squares. 
79 sdev=S (echo "scale-$SC; sqrt($avg2)" | bc) # Square root = 

80 echo $sdev # Standard Deviation. 

81 

92 J- «eelene se? # Rewinds data file. 

83 

84 

85 # # 

86 mean-$(arith mean); count=$? # Two returns from function! 

87 std dev-$(sd $mean $count) 

88 

89 echo 

90 echo "Number of data points in \""Sdatafile"\" = Scount" 

91 echo "Arithmetic mean (average) = $mean" 

92 echo "Standard Deviation = $std dev" 

93 echo 

94 # # 

95 

96 exit 

op 

98 This script could stand some drastic streamlining, 

99 #+ but not at the cost of reduced legibility, please. 
100 
101 
102 TERRA A- A .. A BÀ À À  ] P Óg [L G 4 Ó49 B. B B nà 3M 4 B4B 464.4 
Iu A sample data file (samplel.dat): 


104 
105 Lo 3S 
106 19O 
OW 1.5.1908 
108 omo 
109 18.64 
LAL) 

il 


$ sh sd.sh samplel.dat 


Number of data points in "samplel.dat" = 5 
Arithmetic mean (average) = 18.756000000 
Standard Deviation = .235338054 


PRPRPPRPRPP 
AHN OB WD 


T4RPRREBRRRCRERRRPRERRERRERRRRERRRRERRERRERRRRPRRRRE # 


Example A-38. A pad file generator for shareware authors 


!/bin/bash 
pad.sh 


FE FE TE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE HE FE FE FE FE HE FE HE FE HE HE HE HE HE 
PAD (xml) file creator 

+ Written by Mendel Cooper <thegrendel.abs@gmail.com>. 
+ Released to the Public Domain. 


Generates a "PAD" descriptor file for shareware 
+ packages, according to the specifications 

ar (ue ime ASP 
http://www.asp-shareware.org/pad 

TERRE EE HHH HE EEE ERE HE EEE RE EE HEE HE EEE RE EE HE HE EE 


Accepts (optional) save filename as a command-line argument. 
3E [ —3 uS u ] 


=) ony (nb des (GS) IS) [SP 4o» (eer — ony Gal dES (o5) INSP d 


18 
19 
20 
Zu 
22 
23 
24 
AS) 
26 
27 
28 
29 
30 
E 
32 
33 
34 
39 
36 
3 
38 
EX 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Syl 
52 
53 
54 
35 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
qi 
12 
T3 
74 
75 
76 
3 
78 
y 
80 
81 
82 
83 


then 
savefile-$1 
else 
savefile-save file.xml # Default save file name. 
ial 
# ===== PAD file headers ===== 


HDR1="<?xml version=\"1.0\" encoding=\"Windows-1252\" ?>" 
HDR2="<XML_DIZ_INFO>" 

HDR3="<MASTER_PAD_VERSION_INFO>" 

HDR4-"Nt«MASTER PAD VERSION»21.15«/MASTER PAD VERSION»" 
HDR52"Nt«MASTER PAD INFO»Portable Application Description, or PAD 

for short, is a data set that is used by shareware authors to 

disseminate information to anyone interested in their software products. 
To find out more go to http://www.asp-shareware.org/pad«/MASTER PAD INFO»" 
HDR6-"«/MASTER PAD VERSION INFO»" 


# 
ial ILI satis AO 
{ 
AE [ — "$2" ] 
then 
Seino. —s USE # Get user input. 
else 
Seng =i WS 922 W sr ACCAC Omal cery? 
ie aL 
read var # May paste to fill in field. 
# This shows how flexible "read" can be. 
mie || -— "Eu ] 
then 
echo -e "NtNt«S1 /»" >>Ssavefile # Indent with 2 tabs. 
TESTUEN 
else 
echo -e "\t\t<$1>$var</$1>" >>$savefile 
return S{#var} # Return length of input string. 
ia 
} 
check field length () # Check length of program description fields. 
( 
# $1 = maximum field length 
# $2 = actual field length 
TE [ WE OE We ul ] 
then 
echo "Warning: Maximum field length of $1 characters exceeded!" 
ieaL 
} 
clear # Clear screen. 
echo "PAD File Creator" 
cho Ww " 
echo 
# Write File Headers to file. 


echo SHDR1 »$savefile 
echo SHDR2 >>Ssavefile 
echo SHDR3 >>S$savefile 
echo -e SHDR4 >>Ssavefile 
echo -e SHDR5 >>Ssavefile 
echo SHDR6 >>Ssavefile 


84 


ps 
«o 


4 Company Info 
echo "COMPANY INFO" 


CO 


| HDR-"Company. Info" 


echo "«$CO HDR»" >>Ssavefile 


Hes 
ira 
ia 
iE SL 
iE aL 
fra 
iE aL 


Se SE OE SHE 


feat 


ell 


1l in Company. Name 

ll in Address 1 

ll in Address 2 

1l in City. Town 

ll in State Province 
ll in Zip Postal Code 
LI 3m Corries 


If applicable: 

fill in ASP Member "[Y/N]" 
fill in ASP Member Number 
aii bm MGC Memos Urn 


1l in Company WebSite URL 


ear # Clear screen betw 


n sections. 


Contact MEO 


echo "CONTACT INFO" 
CONTACT HDR-"Contact Info" 
echo "<SCONTACT_HDR>" »»$savefile 


ira 
it 
fta 
Haale 
ira 
iE 31 


ll in Author First Name 
ll in Author Last Name 
ll in Author Email 

ll in Contact First Name 
ll in Contact Last Name 
ibd akin, Comnecace Jona JL 


echo -e "\t</SCONTACT_HDR>" 


ell 


END Contact_Info 
ear 


# Support_Info 


echo "SUPPORT INFO" 
SWPP ORME isi Rk—/Sieyoyoroueie EEEO 
echo "<SSUPPORT_HDR>" >>S$sav 


PL 
fta 
ít 
feat 
Haale 
ira 
iB 41 
ee 


ec 


lb 38: Saless mied. 
LIL a Syeyeyexostit, Mnal 
ll in General Email 
ll in Sales Phone 
JL1L abo. Swjsyooieit_Piavoine 
ll in General Phone 
ll in Fax Phone 
mg —e wyEE/SSUIPIEOEU IIDIS- 
END Support Info 


ho "«/$CO HDR»" >>Ssavefil 


END Company Info 


clear 


# Program Info 

echo "PROGRAM INFO" 

PROGRAM HDR-"Program Info" 
echo "«$PROGRAM HDR»" >>Ssav 


itat 
Haale 
iE aL 
FE 
ies 
cfi 


ll_in Program_Name 

ll in Program Version 

ll in Program Release Mont 
ll in Program Release Day 
ll in Program Release Year 
Jb slim Progen Cose. Dollars 


>>Ssavefile 


efile 


>>Ssavefile 


© 


efile 


h 


150 
ILS Al 
152 
15,3) 
154 
155 
156 
1153 
158 
159 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
ETA 
272 
LHS) 
174 
IS 
176 
Lay 
178 
HS 
180 
181 
182 
183 
184 
185 
186 
187 
188 
18 9) 
190 
3L ent 
192 
193 
194 
LDS) 
196 
197 
198 
1199; 
200 
201 
202 
203 
204 
205 
206 
207 
208 
209 
210 
BALA 
2112 
21L3 
214 
215 


IGLILIL us. Proca Cost (Ole 
fill in Program Type 
fill in Program Release Status "[Beta, Major Upgrade, 
iil ain Procera Joel Supe 

fill in Program OS Support "[Win9x/Win2k/Linux/etc.]" 
fill in Program Language "[English/Spanish/etc.]" 


echo; echo 


# File Info 
echo "FILE INFO" 


"[Shareware/Freeware/GPL]" 


FILEINFO HDR-"File Info" 


echo "<SFILEINFO_HDR>" 


>>Ssavefile 


fill_in Filename_Versioned 
fill in Filename Previous 
fill in Filename Generic 


fill in Filename Long 
fill in File Size Bytes 


fill in File Size K 


iat Jk ue teak ey Sirve Is 
echo -e "\t</S$FILEINFO_HDR>" >>S$savefile 


# END File Info 


# Expire Info 
echo "EXPIRE INFO" 


echo "<SEXPIRE_HDR> 


TESLILIL dug: Bes gone LG 
Expire Count 
Expire Based On 

Expire Other. Info 


Expire Mont 
Expire Day 
Expire Year 


Fh 
H 
= 
= 
ee Mene pes Msg Vo 
5 
RI ER DOLO EE NESA VES 


n 


echo -e "\t</SEXPIR 


# END Expire Info 


# More Program Info 


EXPIRE HDR-"Expire Info" 
>>Ssavefile 


"Y/N" 


E HDR»" >>Ssavefile 


echo "ADDITIONAL PROGRAM INFO" 

fill in Program Change Info 

fill in Program Specific Category 
fill in Program Categories 

fill, in Includes JAVA VM "[Y/N]" 
fill, in Includes VB Runtime "[Y/N]" 
IGLILIL omaes 3üs redes zexeoS «AANA 


Ej 


echo "</SPROGRAM_HDR>" 


# END Program Info 


clear 


AANG="English" 


ecno 


Program Description 
echo "PROGRAM DESCRIPTIONS" 

PROGDESC HDR-"Program Descriptions" 
echo "«S$PROGDESC HDR»" 


ND More Program Info 


>>Ssavefile 


>>Ssavefile 


eeno VKSLANESY SSC seweue i le 


fill_in Keywords "[comma + space separated]" 


SEC, 


ZIG echo "45. 80, 250, 450, 2000 wel progiem ekxeuescastst omes 

217 echo "(may cut and paste into field)" 

218 # It would be highly appropriate to compose the following 

2119) sae Weiner DeSCY fireless wien a wesc echiiceie, 

220 #+ then cut-and-paste the text into the answer fields. 

221 echo 

222 echo " | 45 characters iU 
22/5 1psblbll abies (Cloweuc. IDyexsve dis) 

Daal Cheek Field lengen AS UE 

225 echo 
DIAS d:1LIL 359v Ghar DESE E0 

DAY Check risked lengten BO "go 
228 
Zo Tullo APC ihe eS 230 

230 Cliche sreli dieyetegslo. 250 Wem 
2 
29/2 diesLl1LIL 3b (Cloxeue Desie 450) 
ZS TAA a Oen Desc capu 
234 
235 echo "«/SLANG»" >>Ssavefile 

236 echo "</SPROGDESC_HDR>" >>Ssavefile 
237 # END Program Description 

238 
239 clear 

240 echo "Done."; echo; echo 

24 exeo "Sene tile seg \Wsaweti lew \e 
242 

243 exit 0 


Example A-39. A man page editor 


!/bin/bash 
maned.sh 
A rudimentary man page editor 


Version: 0.1 (Alpha, probably buggy) 

Author: Mendel Cooper <thegrendel.abs@gmail.com> 
Reldate: 16 June 2008 

License: GPL3 


savefile= # Global, used in multiple functions. 
E_NOINPUT=90 # User input missing (error). May or may not be critical. 
=========== Markup Tags ============ # 


TopHeader=".TH" 
NameHeader=".SH NAME" 
SyntaxHeader=".SH SYNTAX" 
SynopsisHeader=".SH SYNOPSIS" 
InstallationHeader=".SH INSTALLATION" 
DescHeader=".SH DESCRIPTION" 


NPRPRPPRPP PP PY 
O (o 0» -1 O Ui i$ (). M P. O xo 0 -1 O O1 4 CQ Io E 


21 OptHeader-".SH OPTIONS" 

22 FilesHeader-".SH FILES" 

23 EnvHeader-".SH ENVIRONMENT" 
24 AuthHeader-".SH AUTHOR" 

25 BugsHeader-".SH BUGS" 

26 SeeAlsoHeader-".SH SEE ALSO" 
2.7] YeYonnjpy— V c dej 


28 4 Add more tags, as needed. 

29 4 See groff docs for markup meanings. 

30 # # 
Sil 


moie EU 

exer «i 

34 clear # Clear screen. 
35 echo "ManEd" 

SG. Gxelag. Vasa y 

37 echo 

38 echo "Simple man page creator" 

39 echo "Author: Mendel Cooper" 

40 echo; echo; echo 

41 } 

42 

43 progname () 

44 ( 

45 echo -n "Program name? " 

46 read name 

47 

48 echo -n "Manpage section? [Hit RETURN for default (\"1\") ] " 
49 read section 


50 ise | v VWSsecenem ] 

S then 

52 section=1 # Most man pages are in section 1. 

53 P 

54 

55 zie [| e V nemet ] 

56 then 

57 savefile=""Sname"."Ssection"" Filename suffix = section. 
58 echo -n "$1 " >>Ssavefile 

S59 namel-$ (echo "Sname" | tr a-z A-A) Change to uppercase, 
60 + per man page convention. 
61 echo -n "$namel" >>Ssavefile 

62 else 

63 echo "Error! No input." Mandatory input. 

64 exit $E NOINPUT Crne an 

65 ít 

66 

67 echo n " \"Ssection\"">>S$savefile Append, always append. 
68 

69 echo -n "Version? " 

41) read ver 

WAL echo -n " \"Version $ver \"">>Ssavefile 

72 echo >>Ssavefile 

73 

74 echo -n "Short description [0 - 5 words]? " 

15 read sdesc 

76 echo "SNameHeader">>Ssavefile 

Vi echo ""SBOLD" "Sname"">>Ssavefile 

76 cho "\- "Ssdesc"">>Ssavefile 

78 

e j 

81 

G2. scat J cab (63 

83 ( # This function more or less copied from "pad.sh" script. 

84 xolg) cim VEZ DW # Get user input. 

85 read var # May paste (a single line only!) to fill in field. 
86 

87 ade (p e "Ewa ] 

88 then 

89 scho “Si Y s»S9ewetile 

90 echo -n "Svar" »»5$savefile 

91 else # Don't append empty field to file. 

92 return $E NOINPUT # Not critical here. 

93 ie aL 

94 

95; echo >>Ssavefile 

96 


Or} 


100 end () 

WO di 

102 clear 

103 echo -n "Would you like to view the saved man page (y/n)? " 
104 read ans 


MOS, aude. qp eua "ex Ungu. exer us egaeiut es MANE tehen exit 3E dE 

106 exec less "$savefile" # Exit script and hand off control to "less" 
107 #+ ... which formats for viewing man page source. 
108 } 

109 

110 

111 # # 

1L3L2. etare 

113 progname "$TopHeader" 

114 fill in "$SynopsisHeader" "Synopsis" 

115 fill in "$DescHeader" "Long description" 

116 # May paste in *single line* of text. 

117 fill in "SOptHeader" "Options" 

abe) desLILJD bg: Re S Heade T  hpsbibesger 

WAS sed 3b: VEA aSa Een  — Uie oue 

120 fill in "SBugsHeader" "Bugs" 

12 Gai jin Se cAc oH eade US also" 


122 fill in "$OtherHeader" ... as necessary. 

123 end # ... exit not needed. 

124 # 

125 

126 Note that the generated man page will usually 

127 #+ require manual fine-tuning with a text editor. 

1279 However, it's a distinct improvement upon 

129 #+ writing man source from scratch 

130 #+ or even editing a blank man page template. 

it St 

132 The main deficiency of the script is that it permits 
133 #+ pasting only a single text line into the input fields. 
134 This may be a long, cobbled-together line, which groff 
S will automatically wrap and hyphenate. 

136 However, if you want multiple (newline-separated) paragraphs, 
137 #+ these must be inserted by manual text editing on the 
138 #+ script-generated man page. 

139) lnpxeiensie (Clie uie) Iwas iclaalis 

140 

141 This script is not nearly as elaborate as the 

142 #+ full-featured "manedit" package (http://wolfpack.twu.net), 
143 #+ but it's much easier to use. 


Example A-40. Petals Around the Rose 


1 #!/bin/bash -i 
2 petals.sh 
3 
4 GEREA4HRRHRERERHERERERERERETETAERSASESEASETETESASAHRSREAAESEAYUASARHASRSEARAHASATEVTYT 
5 Petals Around the Rose 
6 
y Version 0.1 Created by Serghey Rodin 
8 Version 0.2 Modded by ABS Guide Author 
9 
10 License: GPL3 
Lil Used in ABS Guide with permission. 
HEHE EEE HE EEE HE HE EEE EE EE HEE EE EEE EE HE HE EE ER ER HERE EEE EE HEH HE EE HE EH HF 


PR 
w N 


14 
15 
16 
17 
18 
19) 
20 
21 
22 
23 
24 
215 
26 
2 
28 
219 
30 
Sl 
32 
33 
34 
39 
36 
3 
38 
EI 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
Syl 
52 
53 
54 
35 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
qa 
72 
13 
74 
1/5 
76 
p 
78 
9 


hits=0 # Correct guesses. 
WIN=6 # Mastered the game. 
ALMOST=5 # One short of mastery. 
EXIT=exit # Give up early? 
RANDOM-$$ # Seeds the random number generator from PID of script. 
Bones (ASCII graphics for dice) 
bonel[1]-" Ui 
bonel[2]=" @ |" 
bonel[3]=" e " 
bonel[4]-2"| o gi | 
bonel[5]-2"| o ge |# 
bonel[6]-2"| o e |" 
bone2[1]=" © H 
bone2[2]=" i% 
bone2[3]=" © uj 
bone2 [4]=" y 
bone2[5]=" © n 
bone2[6]="| o @ | 
bone3[1]=" v 
bone3[2]="| o y 
bone3[3]="| o y 
bone3[4]="| o e. |" 
bone3[5]="| o ey | 
bone3[6]="| o e Jw 
Boneu poo o yr 
# Functions 
LMSiccUGELoms (0) 1 
clear 
eho ia) UDa yor mecc alingiciebiciesloimse hum) Vp. reac) has 
aise | "Suo. SUA ey Ue ce UP qug. EE 
clear 
echo -e 'NE[34;47m' # Blue type. 
# “cat document" 
cat <<INSTRUCTIONS ZA 
The name of the game is Petals Around the Rose, 
and that name is significant. 


Five dice will roll and you must guess the "answer" for each roll. 
It will be zero or an even number. 

After your guess, you will be told the answer for the roll, but 
that's ALL the information you will get. 


Six consecutive correct guesses admits you to the 
Fellowship of the Rose. 
INSTRUCTIONSZZZ 


echo -e 


NOIS Omi” + Wein Cutie lodge. 


else clear 


Henn 


fortun 
{ 
RANG 
FLOO 
numb 


e | 


E=7 


er=0 


[Es 
«o 


while [ "$number" -le $FLOOR ] 
do 
number=$RANDOM 
let "number %= SRANGE" $ed = 6, 
done 
return Snumber 
} 
throw () { # Calculate each individual die. 
ite boa epg Bil=S? 
iubes W282 
ituew Dude BOSS? 
rorta; BASS? 
FOLENS; 1-99 
calc () { # Function embedded within a function! 
aise, JSI abl 
3) ) rose-2;; 
5 ) rose=4;; 
b ) rose=0;; 
esac # Simplified algorithm. 
# Doesn't really get to the heart of the matter. 
return Srose 
} 
answer=0 
calc "SB1"; answer=$ (expr Sanswer + $(echo $?)) 
calc "SB2"; answer=$ (expr Sanswer + $(echo $?)) 
calc "SB3"; answer=$ (expr Sanswer + $(echo $?)) 
calc "SB4"; answer=$ (expr Sanswer + $(echo $?)) 
calc "SB5"; answer=$ (expr Sanswer + $(echo $?)) 
} 
game () 
{ # Generate graphic display of dice throw. 
throw 
echo -e "\033[1m" EROINA 
acho se nt 
cho "Sbone\tSbone\t$bone\t$bone\t$bone" 
echo -e \ 
"S {bonel [$B1] }\t${bonel [$B2] }\t${bonel [$B3] }\t${bonel [$B4] ) N£$(bone1[$B5] ) " 
echo -e \ 
"$(bone2[$B1])Nt$(bone2[$B2] ) t$ (bone2[$B3] }\t$ (bone2[$B4] }\t${bone2[$B5]}" 
echo -e \ 
"$(bone3[$B1])Nt$(bone3[$B2] }\t$ (bone3[$B3] }\t$ (bone3[$BA4] }\t${bone3[$B5]}" 
cho "$boneNt$boneNt$boneNt $boneNt$bone" 
Sno =e "Nee WE Vie” 
echo -e "X033[0m" s? ‘Weer cic doxes 
cho -n "There are how many petals around the rose? " 
} 
# 
instructions 
wWueLIke [| Weed" LS Wistar ay | # Main loop. 
do 
game 
read petal 


146 Echa "Sere" || aee [9-9] cev meii i JexblWr(eus response Cor (hip. 

147 # Otherwise just roll dice again. 
148 se [p Yeru see © T # If-loop #1. 

149 then 

L50 if [ "Spetal" == "Sanswer" ]; then # If-loop #2. 

TESTE echo -e "\nCorrect. There are $petal petals around the rose.\n" 
1157 ((U Ims JJ 


154 ise [ USES" seep USWBDNUS je then ur lbi ojo qol 

18515 Sele -e V Xem 3 db e 4" fq V Red type. 

156 echo -e "\033[1m" Bold. 

IS) 7 echo "You have unraveled the mystery of the Rose Petals!" 
158 cho "Welcome to the Fellowship of the Rose!!!" 

115S) echo "(You are herewith sworn to secrecy.)"; echo 

160 echo -e "N033[0m" Turn off red & bold. 

LEN break Prane 

162 else echo "You have Shits correct so far."; echo 


164 sue [p VSinnwes seep VSAM T s chan 
165 echo "Just one more gets you to the heart of the mystery!"; echo 
166 iat 


168 pial # Close if-loop #3. 


170 else 

131 echo -e "\nWrong. There are Sanswer petals around the rose.\n" 
172) hits-0 # Reset number of correct guesses. 

3373 ÍE3L # Close if-loop #2. 


17/5 echo =n "Hit ENTER for the next roll, or type \"exit\" to end. " 
176 read 

i7 if [ "SREPLY" - "SEXIT" ]; then exit 

178 fault 


180 iat # Close if-loop #1. 


183 done # End of main (while) loop. 

185 ### 

187 exit $? 

189 # Resources: 

1) http://en.wikipedia.org/wiki/Petals Around the Rose 
(Wikipedia entry.) 


2) http://www.borrett.id.au/computing/petals-bg.htm 
(How Bill Gates coped with the Petals Around the Rose challenge.) 


Example A-41. Quacky: a Perquackey-type word game 


!/bin/bash 
qky.sh 


FEAE FE FE E TE FE FE HE E TE FE FE HE TE EH HE EE EEE EE EE FE AE EE EEE EE EE ER RE EE EEE E E EE E E E EE EEE 
QUACKEY: a somewhat simplified version of Perquackey [TM]. 


Author: Mendel Cooper <thegrendel.abs@gmail.com> 

version 0.1.02 03 May, 2008 

License: GPL3 

HEE FE E HHH HE EEE HE EE HE HE EE EEE EE EEE HE EE ERE RE EE E E E HE E E EEE EE EE HEE HE EH 


S W9 @ A oy oS (e9 N [2 
dE Je db db db d 


[^ 


Mop p op pop EPP PB YE 
O (9 0 -10 Ori QN D 


(ES) (ESF dS TSOP RS) ISSR Ry (R5 TRO INS) ARS) 
i Sp Me) Gere oy, (Gal ode Gey NS) TS 


WWWW CO CO CO CO 
Ge) er X] (xv Gal GSS (63. [RS 


A e 
= €» 


QQ Pf Pe SP um um aum 
(cy We) Cs) p ony Cal des Cd) 9) 


C1 Ol 
NO 


(om Gal (Gm Xn Conr (Gur 6 
Wey (oer «| py (Qn eS (99 


Or OV OY OF OF OY O 
oy Ga de es) fo) [^ 8» 


Sl Leph Tp) OY 
(Sy Moya} 3] 


-1 
[Em 


ay ys -—d 
(ox. (Ont ge Gy IS) 


WLIST-/usr/share/dict/word.lst 


AA^A^AA^A^A^^^ 


or 


Se oce OSE cR H H 


NONCONS=0 
CONS 
SUCCESS=0 


# 
# 


ASEE Wore SL 


one word per line, 
A suggested list is the script author's 
http://bash.neuralshortciruit.com/yawl-0.3.2.tar.gz 


Word list file found here. 


UNIX format. 
"yawl" word list package. 


heros / / r enol se), orero Moses: TLabloxsu/ Seil (0) o lo 2 s TEcae o (1, 


Word not constructable from letter set. 
Constructable. 


Zero out value of letter 
Minimum word length. 
Maximum number of words in a given category. 


General-purpos 


(Sue dt QoybuaYOL) .; 


penalty for unacceptable words. 


Duplicate word error. 


Time for word input. 


10 letters for non-vulnerable. 
13 letters for vulnerable 


# Letter distribution table shamelessly borrowed from "Wordy" game, 
written by a certain fine fellow named Mendel Cooper. 


NVLET-10 

VULET=13 

declare -a Words 
declare -a Status 
declar a Score-( 
letters-(an sr 
eyerefegt 
WW gt 2 ih 4o) xw wy vy © 
jig Gr ie ak YE db V me WU 
Tob Car 19927 
declare -a LS 


iE 
El 
S 


V 


numelements=S {#letters[@] } 


randseed-"$1 


instructions 


{ 


clear 


echo "Welcome to QUACK 
echo -n "Do you need instructions? 


alice’ ell 
clear 
echo -e 


J Sieg] 


0 


"NJ 


cato €xINSI 


QUACKEY is a variant of Perquackey 


oO > 3 
© EE E 


n 


The rules ar 


but it is otherwis 


iE 


N 


UA =o 


E[31;47m' 
RUCTION1 


same, 


k 
al 
n 


o 


(9 tar o) 
(er den- de dj 


X 


EY, 


ito} d» is 
$00 
AO H H- 


m 


the anagramming word construction game."; 


Sergi] 


E 


"ny" lg 


0 (9 (9 Q 9 9 OOO Q Q9 ) 


(y/n) 


(not yet implemented). 


# Red foreground. 


EEMIE 


then 


, 


DN 


read ans 


E[34;47m' 


but the scoring is simplified 
and plurals of previously played words are allowed. 
"Vulnerable" play is not yet implemented, 


As the game begins, 


featur 


complet 


the player gets 10 letters. 
The object is to construct valid dictionary words 


of at least 3-letter-length from the letterset. 


Each word-length category 


3-letter, 


4-letter, 


5-letter, 


fills up with the fifth word entered, 


for blue. 


echo 


[s 
«o 


and no further words in that category are accepted. 


The penalty for too-short (two-letter), duplicate, unconstructable, 
and invalid (not in dictionary) words is -200. The same penalty applies 
to attempts to enter a word in a filled-up category. 


INSTRUCTION1 

exeo. —1m Wise HAND TOI MEE JSCS (Qut ae TRUET LONS Wp eee ail 

cat <<INSTRUCTION2 

The scoring mostly corresponds to classic Perquackey: 
The first 3-letter word scores 60, plus 10 for each additional one. 
The first 4-letter word scores 1207 prus 20 for each additional one. 
The first 5-letter word scores 200, plus 50 for each additional one. 
The first 6-letter word scores 300, plus 100 for each additional one. 
The first 7-letter word scores 500, plus 150 for each additional one. 
The first 8-letter word scores 150, plus 250 tor sac elei. ome, 
The first 9-letter word scores 1000, plus 500 for each additional one. 
The first 10-letter word scores 2000, plus 2000 for each additional one. 
Category completion bonuses are: 


3-letter words 100 

4-letter words 200 

5-letter words 400 

6-letter words 800 

7-letter words 2000 

8-letter words 10000 

This is a simplification of the absurdly complicated Perquackey bonus 
Scoring system. 


INSTRUCTION2 


echo =m Viie “TINA for tinal gage GE meci wet aos, ME ral azil 


cat ««INSTRUCTION3 


Hitting just ENTER for a word entry ends the gam 


Individual word entry is timed to a maximum of 10 seconds. 
*** Timing out on an entry ends the game. *** 
Other than that, the game is untimed. 


Game statistics are automatically saved to a file. 


For competitive ("duplicate") play, a previous letterset 
may be duplicated by repeating the script's random seed, 
command-line parameter \$1. 

For example, "gqky 7633" specifies the letterset 

(m cl sb dE ze in wi. gy ik 


INSTRUCTION3 
echo; echo -n "Hit ENTER to begin game. "; read azl 
echo -e "\033[0m" s rurn Quei rec, 
else clear 
ít. 
clear 


143 


144 

145 seed random () 

IAG # Seed random number generator. 
AAS if [ -n "Srandseed" | # Can specify random seed. 

148 then #+ for play in competitive mode. 
149 # RANDOM="$randseed" 

150 echo "RANDOM seed set to "Srandseed"" 

ESAL else 

152 randseed-"$$" # Or get random seed from process ID. 
153 echo "RANDOM seed not specified, set to Process ID of script ($$)." 
154 3E aL 

155 

156 RANDOM="Srandseed" 

11577) 

158 echo 

159 j 

160 

toi 

162 get_letset () 

Les 1 

164 element=0 

165 cho -n "Letterset:" 

166 


167 for lset in $(seq SNVLET) 

168 do 4 Pick random letters to fill out letterset. 
169 LS [element ]="${letters[$((RANDOM%numelements))]}" 
170 ((element++) ) 

Ji al done 

T2 

7S echo 

174 exo: "Spe [T i 

JU JS 

i75 Y 

Aon 

HI 

LTO dd werd wq) 

IO 4i 

181 wrd-"$1" 

182 local idx=0 

183 

184 Status[0]="" 

185 Status [3]="" 

186 Status[4]="" 


187 

188 wüscLIe [. US onek [ache] py l= vuv m 
189 do 

190 ai | “S44 Kiewels ack} = "Sree! I 
iL 9T. then 

192 Status [3]="Duplicate-word-PENALTY" 
193 let "Score[0]= 0 - SPENALTY" 
194 let "Score[1]-=SPENALTY" 

195 return $E DUP 

196 ita 

1:97) 

LOS ( (bos) )) 

199 done 

200 


201 Words[idx]-"$wrd" 
202 get_score 

203 

204 } 

205 

206 get score() 

207 { 

208 local wlen=0 


[e 


Pee ee Peet 
ee) SJ) vou, Gri dem Gy) RS) 02. Wo) 


NONNNNNNN DN NH 


N N 
[S9 1: 
(y We) 


local score=0 
local bonus=0 
local first_word=0 
local add_word=0 
local numwords=0 


wlen=S$ { #wrd} 

numwords=S {Score [wlen] } 

Score[2]=0 

Status[4]="" sr dlislalipaleillavas “Morons "E (0), 


case "Swlen" in 

3) first_word=60 
add_word=10;; 
4) first_word=120 
add, word-20;; 
5) first word-200 
add word-50;; 
6) first word-300 
add, word-100;; 
7) first word-500 
add, word-150;; 
8) first word-750 
add  word-250;; 
9) first word-1000 
add, word-500;; 
10) first word-2000 


add word-2000;; 4 This category modified from original 


esac 


( (Score [wlen]++) ) 

if [ ${Score[wlen]} -eq SMAXCAT ] 

then # Category completion bonus scoring simplified! 
case Swlen in 


3 ) bonus=100;; 

4 ) bonus-200;; 

5 ) bonus-400;; 

6 ) bonus-800;; 

7 ) bonus-2000;; 
8 ) bonus-10000;; 


esac # Needn't worry about 9's and 10's. 
Status[4]-"Category-$wlen-completion***BONUS***" 
Score[2]-S$bonus 


else 
Status[4]="" # Erase it. 
aak 
let Vecore = Situs vor s $add word * $numwords" 
if [ "$numwords" -eq 0 ] 
then 
Score[0]=Sscore 
else 


Score [0]=Sadd_word 
iE aL # All this to distinguish last-word score 
#+ from total running score. 
let "Score[1] += ${Score[0]}" 
let "Score[1] += ${Score[2]}" 


get word () 
( 


local wrd-'' 


rules! 


(V9) (69. (69) (73) ay (93. 169 fe») 


WW 
Nh + 
€» 3e) 


read -t STIMEOUT wrd # Timed read. 
echo Swrd 


ILS exXoliorsie ieu dere ore (0) 


{ 


# This was the most complex and difficult-to-write function. 
local -a local hnS2( "S(LS[Q]y" ) # Local copy of letter set. 
local is. found-0 

local idx=0 

local pos 

local strlen 

lkexewul dlexemwul tore ( WE y 

strlen-$(flocal word) 


whine [| Viauebe' tenenu] 
do 
le rovnc=S (Eeyore Immela "SIL JS] jV WUSTIoxe. vorc ieks dp" 
su [| Use scowl” See, VSINONCONSY J| sr NOE COnseructablgl 
then 
Scho "SIEBEN P Perv 
else 


letters! 


(os = (Sis rowr = 1» 7 25) # Compensate for spaces betw. 
local LS[pos]-S$NULL # Zero out used letters. 
((idx++) ) # Bump index. 
if aL 
done 


echo “VSSuCCISS” 
return 


we vetis d) 


{ 


# Surprisingly easy to check if word in dictionary 
noe =o "SI UON qs # ... thanks to 'grep' 
echo $? 


check word () 


{ 


aie | x eSI | 
then 

return 
ipa 


Status 
Status 
SIEGE US 
Status 


TS cCons- si (LS. Comeinicneitaloike Sil) 
me | WSaseome ] 
then 
Status [1]="constructable" 
we (Gis valie) Sal) 
ai [ "SU ee USSUCCIHSSV" ] 
then 
Status [2]="valid" 
strlen=$ {#1} 


if [ ${Score[strlen]} -eq "SMAXCAT" ] Category full! 

then 
Status [3]="Category-Sstrlen-overflow-PENALTY" 
return SNG 

teak 


case "Sstrlen" in 


341 
342 
343 
344 
345 
346 
347 
348 
349 
350) 
Sol 
352 
353 
354 
39S 
356 
S 
3368 
259 
360 
361 
362 
363 
364 
SOS) 
366 
367 
368 
369 
SVO 
371 
372 
SUIS 
374 
Sm 
376 
377 
378 
So 
380 
381 
382 
3893 
384 
385 
386 
EM 
388 
389 
39 
39 
392 
393 
394 
395) 
396 
397 
396 
3989 
400 
401 
402 
403 
404 
405 
406 


d x 3 
Status [3]="Two-letter-word-PENALTY" 
return SNG;; 
ZA) 
Status[3]="" 
Mew SSvVCCHSS. E 
esac 
else 
Status [3]="Not-valid—-PENALTY" 
return SNG 
I3 
else 
Status [3]="Not-constructable-PENALTY" 
return SNG 


iE aL 


### FIXME: the above cod 


Streamlin 


display_words () 


{ 


local idx=0 
local wlenO 


clear 
cho "Letterset: 
echo "Threes: 


S (CS (Le I) Jj 7 


Fours: Fives: 


Sixes: 


Sevens: 


Eights: 


Chom 


while [ 
do 
wlen0=$ { #Words [idx] } 
case "$wlenO" in 
39 4&9 

) echo 
) echo 
6) echo 
) 

) 


WS ikon) licks kW de vu] 


echo 
echo 


"S(Words[idx])" 
(ebr ) 


done 


### FIXME: 


play () 


{ 


word="Start game" # Dummy word, to start 


while [ 
do 


US ] d 
#+ then game ends. 

Seng WiSintouccla VS stratte [eU] Ji 

echo -n "Last score: [${Score[0]}] 

total=${Score[1] } 

word=S (get_word) 

check_word "Sword" 


Lie [ 
then 


USO) VESUCCHSS” | 


-eq 


The word display is pretty crude. 


, 


If player just hits return 


TOTAL score: 


mr 


(blank word), 


[S{Seers D 31 3 


, 


Next 


word: 


fs (Em pis du Ss sy ghe. qüm Es iy den PES qus fi qe quo des qs qme GS dex dy qi qim q ex SS dus is SS qe qdEs fes qf ES des ivy quy Se I qs qus ey a qe PES qim dese qus dms quy SSS qi ques SS 


add word "Sword" 

else 
let "Score[0]= 0 - SPENALTY" 
let "Score[1]-=SPENALTY" 

ia 


display_words 
done # Exit game. 


### FIXME: The play () function calls too many other functions. 


### This is perilously close to "spaghetti code" 


end_of_game () 
{ # Save and display stats. 


HHEEREPEPEEPEPEEE EEE EEFAUC OSAVEHETHEFEFEEFEFEEPETEEE EEE ETE 
savefile-qky.save.$$ 
# AA (PID) qr SETHE 
echo ‘date >> S$savefile 
eine "Ier i Sueuneleeel Gesmcelom seach) VSS Ssaweraile 
cho -n "Letterset: " >> Ssavefile 
echo "S{LS[@]}" >> Ssavefile 
how " >> $savefile 
echo "Words constructed:" >> S$savefile 
echo "S{Words[@]}" >> $savefile 
echo >> $savefile 
echo Veronese Süpoxredlu 2> SGsavarile 


echo "Statistics for this round saved in \""Ssavefile"\"" 
FE FE TE FE TE FE FE FE FE FE FE FE TE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE FE HE FE FE FE FE FE FE FE FE FE FE FE FE FE HE HE HE HE HE 


Geng "Sene iter wals Coumes Sroralw 
echo "Words: ${Words[@]}" 


instructions 
Seed random 
get letset 
play 
end of game 


1) Clean up code! 


3) Improve the time-out ... maybe change to untimed entry, 
ar but with a time limit for the overall round. 

An on-screen countdown timer would be nice. 

Implement "vulnerable" mode of play. 


S| Xexy to en 


ais Jones! d 


Reference for more info: 
http://bash.neuralshortcircuit.com/qky.README.html 


Example A-42. Nim 


Prettify the display words () function (maybe with widgets?) . 


Improve save-to-file capability (and maybe make it optional). 


BO ppbbpmpmpnnurt 
O0 -1 OY Ui 4 CQ). 9. IB. O (0 00 -1 O Or dS (IN S 


OSE EO [9r TNO) IS) [AS JOY [9r ISS) [mox Ie) TPT 
[= (€» (we) Ge cr (ox (nh de GH) [SS p C» wo 


C0 CO C0 CO CO CO CO CO 
We) Ce} Si) ony Gal SS. (63) I) 


D e 
[€ 


Cr GE PP PS am aum 
(&») Me) (Gor — (ox Gal dex Goo 9) 


C1 U1 
OP 


(yh (On Gm Gal Gah (On (On 
(Ko) (6e) <j) fe»y (On PS (o3) 


Ov OV OY OV OV 
Hs Qu DS) [L3 c» 


65 
66 


(Gn) WSS igs) IS) [ 


H 


t H 


Hd 


!/bin/bash 
nim.sh: Game of Nim 


Author: Mendel Cooper 
Reldate: 15 July 2008 
License: GPL3 


echo = "Wess" sp vellon 16, , OVAL mech lore. 


cat =<INSTRUCTIONS 


Nim is a game with roots in the distant past. 
This particular variant starts with five rows of pegs. 


he number at the left identifies the row. 


turn consists of removing at least one peg from a 


or example, in row 2, above, the player can remove 
he player who removes the last peg loses. 


2E 
A 
It is permissable to remove ALL the pegs from a row. 
E 
T 


he strategy consists of trying to be the one who r 


ROWS-5 Five rows of pegs (or matchsticks). 
WON=91 Exit codes to keep track of wins/losses. 
LOST=92 Possibly useful if running in batch mode. 
QUIT=99 
peg_msg= Peg/Pegs? 
Rewset OS 43 2 L) # Array holding play info. 
$ (Rows [0] holds total number of pegs, updated after each turn. 


Other array elements hold number of pegs in corresponding row. 


instructions () 
{ 
clear 
tput bold 
cho "Welcome to the game of Nim."; echo 
echo -n "Do you need instructions? (y/n) "; read ans 
Ge [ MiSs eras = RH —Oo Nis eye Ul = TAAL ive then 
clear 


2 Tice" 


he human player moves first, and alternates turns with the bot. 


single row. 


lb By e One 4) SES 


he next-to-last peg(s), leaving the loser with the 


o exit the gam arly, hit ENTER during your turn. 
STRUCTIONS 


echo; echo -n "Hit ENTER to begin game. "; read azx 


echo -e "X033 [0m" # Restore display. 
else tput sgr0; clear 
ie 


clear 


tedbbw up» (()) 


{ 


moves 
final peg. 


let "Rows[0] = ${Rows[1]} + ${Rows[2]} + $(Rows[3]) + ${Rows[4]} + \ 


$(Rows[5])" # Add up how many pegs remaining. 


67 } 
68 


69 
70 display () 
Ta d 
72 index=1 # Start with top row. 
TS echo 
74 
U5 while [ "Sindex" -le "SROWS" ] 
76 do 
773) p=$ {Rows [index] } 
78 echo -n "Sindex: " # Show row number. 
79 
80 # 
81 # Two concurrent inner loops. 
82 
83 indent=Sindex 
84 maaike || Wisisuarckeine” 85 (Qo ] 
85 do 
86 Seino =m Yow # Staggered rows. 
87 ((indent——) ) # Spacing between pegs. 
88 done 
89 
90 maile [p Uso" -ct O | 
91 do 
92 echon m ME o 
23 ((p--)) 
94 done 
95 # 
96 
S echo 
98 ( (index++) ) 
9g done 
100 
101 tally up 
102 
103 rp-$(Rows[0]) 
104 
i95; ake I Wego ca di j 
106 then 
107 peg msg-peg 
108 final msg-"Game over." 
109 else # Game not yet over 
110 peg msg-pegs 
Adi final_msg="" i? o o o SO Vittimedl messes” 3s lol - 
3352 iE ak 
11.9 
114 echo " Srp $peg msg remaining." 
115 echo " We fesuevsitl ey 
116 
11.7) 
118 echo 
i19 j 
120 
121 player_move () 
d. 4E 
12:5] 
124 echo "Your move:" 
11223) 
126 schoo simi Minien icone N 
WZT while read idx 
128 do # Validity check, etc. 
1,259) 
130 abad [^ emos Uke bea T] 4 Hitting return quits. 
ILS then 
132 echo "Premature exit."; echo 


133} 
134 
LSS) 
136 
135) 
138 
13.9 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
dL 
1152 
15.3) 
154 
1155 
156 
15/7 
158 
1.59; 
160 
161 
162 
1,3 
164 
165 
166 
167 
168 
169 
170 
eal 
172 
LHS) 
174 
LHS) 
176 
ETAT 
178 
178; 
180 
181 
182 
183 
184 
185 
186 
ieg 
188 
189 
190 
Lil 
192 
193 
194 
195 
196 
197 
199.) 


Ewe  exeneo) # Restore display. 
exit SQUIT 
ft 


aie || VSilcke gi "UISEOWS" -© “Sarebe —dA 4 i # Bounds check. 
then 
echo "Invalid row number!" 
echo -n "Which row? " 
else 
break 
iral 
# TODO: 
# Add check for non-numeric input. 
# Also, script crashes on input outside of range of long double. 
# Fix this. 


done 


cho -n "Remove how many? " 
while read num 


do # Validity check. 
aie || =2 VS | 
then 
echo "Premature exit."; echo 
tput sgr0 # Restore display. 
exit SQUIT 
i3 
sue [p Uus" gir. SiRews [aub] d so "Su" dX: db o] 
then 


echo "Cannot remove $num!" 
cho -n "Remove how many? " 
else 
break 
iE dL 
done 
# TODO: 
# Add check for non-numeric input. 
# Also, script crashes on input outside of range of long double. 
ur Ease its. 


let "Rows[idx] -= $num" 


display 
tally up 


se dp Sew] -ee p d | 

then 

echo " Human wins!" 

echo " Congratulations!" 
tput sgr0 # Restore display. 
echo 

exit $WON 

1E at 


if [ $(Rows[0]) -eq 0 | 


then # Snatching defeat from the jaws of victory 
echo " ig exl DW 
echo " You just removed the last peg!" 
echo " Bot wins!" 
tput sgr0 # Restore display. 
echo 
exit $LOST 
ia 


200 

201 Dot move i) 

20/2. 4 

203 

204 row b-0 

2/015 while [[ $row b -eq 0 || S(Rows[row b]) -eq 0 ]] 
206 do 

207 row b-S$RANDOM # Choose random row. 
208 let "row b $- SROWS" 

209 done 

210 

2d 1L 

2012. num_b=0 

21 r0=S {Rows [row_b] } 

214 

215 sse [ US" ew di o] 

BAG then 

21.7) num b-1 

218 else 

219 lee "sau 1o = Sie) — iL” 

220 # Leave only a single peg in the row. 
221 fan # Not a very strong strategy, 

222 #+ but probably a bit better than totally random. 
22) 

224 let "Rows[row b] -= $num b" 

DAD acino in Msewe W 

226 echo "Removing from row $row b ... " 

22 

212209) xi [ "Suus oU -e6 d | 

229 then 

230 peg_msg=peg 

2371 else 

232 peg_msg=pegs 

233 itx 

234 

235 echo " $num b $peg msg." 

236 


297 display 
238 tally up 


239 

240 iie [D Sqmeme[]» ee X I 

241 then 

242 echo " BOT warme ts 

243 tput sgr0 # Restore display. 

244 exit SWON 

245 ira 

246 

247 ] 

248 

249 

AO. s # 
25 aei ions 4 If human player needs them 

252 ijo. lee! # Bold characters for easier viewing. 
25.) (hier # Show game board. 

254 

255 while [ true ] # Main loop. 

256 do # Alternate human and bot turns. 


2251) player move 

25) bot_move 

259 done 

260 # # 
291 

262 4 Exercise: 

DIGNUM ae 

264 4 Improve the bot's strategy. 


265) There is, in fact, a Nim strategy that can force a win. 

266 See the Wikipedia article on Nim: http://en.wikipedia.org/wiki/Nim 
267 Recode the bot to use this strategy (rather difficult). 

268 

269 Curiosities: 

QU ip ce———————— 

TAL Nim played a prominent role in Alain Resnais' 1961 New Wave film, 
272 #+ Last Year at Marienbad. 

273 

274 In 1978, Leo Christopherson wrote an animated version of Nim, 

BUS) gir Maeheosel suy, POL idee. IWRS 0) MOCSL Ma 


Example A-43. A command-line stopwatch 


!/bin/sh 
sw.sh 
A command-line Stopwatch 


Author: Padraig Brady 

http://www.pixelbeat.org/scripts/sw 

(Minor reformatting by ABS Guide author.) 

Used in ABS Guide with script author's permission. 

INO 

This script starts a few processes per lap, in addition to 
the shell loop processing, so the assumption is made that 
this takes an insignificant amount of time compared to 

the response time of humans (~.1s) (or the keyboard 
interrupt rate (-.05s)). 

Ue rose (oli must do ntered twice if characters 
(erroneously) entered before it (on the same line). 

'?' since not generating a signal may be slightly delayed 
on heavily loaded systems. 

Lap timings on ubuntu may be slightly delayed due to: 
https://bugs.launchpad.net/bugs/62511 


NPRPRPRPRP PPP PY 
O io 0» -1 O Oi i$ (). M P. O xo 0 -1 O O1 4 CQ I E 


ZA Changes: 

22 V1.0, 23 Aug 2005, Initial release 

25 Wisi, 26 ded 2007; Allow gorn syollsies: anci lape rrom hel 3boswOYe/eE aves c 

24 Only start timer after a key is pressed. 

2:5 Indicate lap number 

26 Cache programs at startup so there is less error 

27 due to startup delays. 

28 V1.2, 01 Aug 2007, Work around 'date' commands that don't have 
nanoseconds. 
Use stty to change interrupt keys to space for 
laps etc. 


Ignore other input as it causes problems. 
V1.3, 01 Aug 2007, Testing release. 
V1.4, 02 Aug 2007, Various tweaks to get working under ubuntu 
and Mac OS X. 
V1.5, 27 Jun 2008, set LANG-C as got vague bug report about it. 


export LANG-C 


CO CO CO CO CO CO CO CO CO CO DO 
© co 3 oO OE WN ow 


40) vinie =e) (0) # No coredumps from SIGQUIT. 

AV ceea UY TUS GF tomora Cieiel—4 Just lm caser 

42 save tty-' stty -g && trap "stty $save tty" EXIT # Restore tty on exit. 
43 stty quit ' ' # Space for laps rather than Ctrl-*. 

4/4 gp" Gxout — "m ae 2 diese Sputs rather than CUCID 

45 gri; =SClN© # Don't echo input. 

46 

47 cache progs() ( 


48 stty » /dev/null 


49 
50 
Sil 
52 
59 
54 
53 
56 
57 
59 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
val 
12 
73 
74 
1/5 
76 
T3 
78 
[i9 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
21 
22 
9s 
94 
95 
96 
97 
98 
9S; 
100 
101 
102 
119.5) 
104 
105 
106 
107 
108 
109 
110 
111 
132 
JL S) 
114 


date » /dev/null 

grep . « /dev/null 

(echo "import time" | python) 2» /dev/null 
bc « /dev/null 

sed '' « /dev/null 

joune Vil S ele miel li 

/usr/bin/time false 2» /dev/null 

cat « /dev/null 


) 


cache progs # To minimise startup delay. 


date +%s.%N | grep -qF 'N' && use python-1 4 If ‘date’ lacks nanoseconds. 
now() { 
ai [ "Uwe isle" ]p- Chen 
echo "import time; print time.time()" 2>/dev/null | python 
else 
Denei WS. Ziel "eee. maso SN 
ia 


fmt_seconds() { 

seconds-$1 

mins= echo $seconds/60 | bc^ 

xi L “Simla = VOW je leu 
seconds= echo "Sseconds - ($mins*60)" | bc 
echo "Smins:$seconds" 

else 
echo "Sseconds" 

iE aL 


Eora (4 
end-'now' 
total-'echo "Send Sensus" || lex 
fmt seconds $total 


stop() ( 
[ "Slapped" ] && lap "Slaptime" "display" 
total 
exit 


lap() { 
lajsicime=" acl "Sil" | ged an "ms reall Ng 19 \ (22 \) ALA 
[ ! "Slaptime" -o "$laptime" = "0.00" ] && return 
# Signals too frequent. 
laptotal= echo Slaptime+OS$laptotal | bc^ 
ae [ VS2Z e Wende 15 xen 
lapcount-' echo OS$lapcount+1 | be 
laptime-' fmt seconds $laptotal' 
echo $1aptime "($lapcount)" 
lapped="true" 
laptotal="0" 
ie al 


Sehe =m VSjoaee itor lap | 2 itor pis | Ctrl-C to strosa | Spese t9 Stat... $2 


while true; do 
trap true INT QUIT # Set signal handlers. 
laptime-'/usr/bin/time -p 2>&1 cat »/dev/null^ 
ret-$? 
ieeja V ^ JUNE OUI # Ignore signals within this script. 
ic [ Sree Gp i =e Srat er 2 —g Sue q 130 ]; then # SIGINT = stop 
[ | "Ssswsusr" ] ee 4 eoo Save esoupg ] 


ALS) stop 


LG elif [ $ret er 9 =o Suet er 3LSUL 1g Elekeum # SIGQUIT = lap 
10807) aie dp d. weReuet" je Tte 

IES) start= now || exit 1 

TLO echo >&2 

120 continue 

TEE Eal 

122 lap "Slaptime" "display" 

11273) else # eof = split 

2A [ T wSenesue T ds eE EE 

12:5) total 

LAG lap "Slaptime" # Update laptotal. 
T27 ita 

128 done 

112: 


130 dex f 


Example A-44. An all-purpose shell scripting homework assignment solution 


! /bin/bash 

homework.sh: All-purpose homework assignment solution. 

Author: M. Leo Cooper 

If you substitute your own name as author, then it is plagiarism, 
+ possibly a lesser sin than cheating on your homework! 

License: Public Domain 

This script may be turned in to your instructor 
+ in fulfillment of ALL shell scripting homework assignments. 


Mop pop op op PPP PY 
O (o 0» -1 O Oi i$ (). M P2. O xo 0 -1 O O1 i CQ I 2 


ju d (y 

( 

40 echo -n "S{alph[$1]}" 
41 acing =m G Ug 

42 sleep $DLA 

43 e) 


It's sparsely commented, but you, the student, can easily remedy that. 
The script author repudiates all responsibility! 

DLA=1 

P1=2 

P2=4 

P3=7 

PP1=0 

PP2=8 

AXL=9 

E_LZY=99 
2L 
22 declare -a L 
23 mgl" 4 Qg Ly 29 Q 13 Le 39 17 2 2 39 34 ay 26 
24 jin[i]e"g 29 12 14 is 19 29 4 12 15 y Q9 19 S9 2 (0) di ii 24 29 I7 4 6 LA rge 
25 15[2]-2929 19 7 gg 39 29 8 29 7 Q9. 21 4 29) is 42 (& iil 4 2 ig & By 
26 m3]"19 da 29 2 14 12 15 11 4 X39 4 29 X39 7 $$ is 29" 
27 WAS 2 y 14 a4) Wi 22 34 17 LO 29 0 1G de & 6 19 12 4 19 19 26" 
28 if Hj Svs ti 4 © gd 29 0 2 2 4 is 19 29 12 24 29 7 20) 142 3 aid 4 29" 
29 m -Uwa 759 2 20 igs 24 29 344 5 29.4 (6$ y 4 (5 i id 2 is 29w 
30) iy esp 0). 25 By vs} A dusk dies 2710 
Sb ame Soy us Sy 2S). Ge aly) Sk SP IL 2: 02$) (9) ay a 9r GE 4:949 Lal 220 25 
S2 asp9]ewiv- 7 O 3 39 29 24. X34 20 26V 
33} 
34 declare -a \ 
35 culpe AB CD ie BE lek 2b Jp d mh IMINO 2 QI S kU WW Ok X £4 oe, go " V 
36 
E 
38 
3S 


44 
45 
46 ( 
47 
48 } 
49 
50 
UE 
52 
58 
54 } 
55 
56 restore 
37. 4 
58 
59 
60 } 
61 
62 
63 
64 ( 
65 

66 

67 

68 

G9 } 
70 

71 d 
"72 19 
73 
74 
V5 
76 
77) 
TÈ 
79 
80 
81 
82 
83 
84 
85 
86 restore 
87 # 
88 
89 
90 
91 
97 
93 # A typical example of an obfuscated script that is difficult 
94 #+ to understand, and frustrating to maintain. 

95 # In your career as a sysadmin, you'll run into these critters 
96 #+ all too often. 


E[31;48m\033[1m' 


eho =- Vya! 
sleep SDLA 


O 


echo -e 'X033[0m' 
tput sgr0 


# Bold off. 
# Normal. 


oie Juge aia Sil 
do 

joe jhe 
done 


UG ese 


Je 


for i in $(seq 0 SMAXL) 
do 
jg "eds 
sag [Dp ea 99 
then 
Que 
elif 
then 
(Chap 
i3 
done 


MISTER E ung -eq Sey) N LUST UU -eq NISI 1] 


ii menm -eq Ji seien o | | MISERE -eq "$pp2" ji 


echo 


exit SE LZY 


Example A-45. The Knight's Tour 


#!/bin/bash 
# ktour.sh 


yy (nl dex GS) [m (5 


# author: 
reldate: 
# license: 


+ 


mendel cooper 
12 Jan 2009 
public domain 


(Not much sense GPLing something that's pretty much in the common 
* domain anyhow.) 


7 
8 
9 

10 dHERSESHTAEAd3HTHERETHRESEA3HESERETAHTARHARHSRESEAUETAEASHRASREREASRSRESRHTSETETHT 
allele The Knight's Tour, a classic problem. 

12 

13 The knight must move onto every square of the chess board, 

14 but cannot revisit any square he has already visited. 

15 

16 And just why is Sir Knight unwelcome for a return visit? 

17 Could it be that he has a habit of partying into the wee hours 

18 #+ of the morning? 

Lg Possibly he leaves pizza crusts in the bed, empty beer bottles 

20 #+ all over the floor, and clogs the plumbing. 

21 

22 

23 

24 Usage: ktour.sh [start-square] [stupid] 

25 

26 Note that start-square can be a square number 

ZI sar alm "Ele. ieeunge: 0 = 63) ua. Que 

28 a square designator in conventional chess notation, 

28 such as al, f5 nay etc. 

30 

Salt If start-square-number not supplied, 

32 #+ then starts on a random square somewhere on the board. 

33 

34 "stupid" as second parameter sets the stupid strategy. 

35 

36 Examples: 

37 iieoxue.s elm 2S} starts on square #23 (h3) 

38 ktour.sh g6 stupid starts on square #46, 

39 using "stupid" (non-Warnsdorff) strategy. 

40 HEHEHE EE EERE HEH EH HE EEE EE EE HH EE EEE EE EE HE HE ERE EE HEHE EERE EE HH HEH 

41 

42 DEBUG= # Set this to echo debugging info to stdout. 

43 SUCCESS=0 

44 FAIL-99 

45 BADMOVE--999 

46 FAILURE-1 

47 LINELEN-21 # How many moves to display per line. 

48 # # 

49 # Board array params 


50 ROWS=8 # 8 x 8 board. 

51 COLS=8 

52 let "SQUARES = SROWS * SCOLS" 

5S llete UMAX = SOURS) = I 

54 MIN=0 

55 # 64 squares on board, indexed from 0 to 63. 

56 

57 VISITED=1 

58 UNVISITED--1 

59 UNVSYM="##" 

60 # # 

61 # Global variables. 

62 startpos= # Starting position (square #, 0 - 63). 
63 currpos= # Current position. 

64 movenum= # Move number. 

65 (RUE IBEOISESS 7/ # Have to patch for £5 starting position! 
66 

67 declare -i board 


68 4 Use a one-dimensional array to simulate a two-dimensional one. 

69 # This can make life difficult and result in ugly kludges; see below. 
70 declar i moves # Offsets from current knight position. 

pr 

72 


73 initialize board () 


74 
1/53) 
76 
T3 
78 
WS 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
gi 
22 
DI 
94 
95 
96 
2 
98 
9e; 
100 
101 
LOZ 
103 
104 
105 


m 
«o 


{ 


lo 


fo 
do 


do 


cal idx 

we ebs suid 40005 ,63)}! 
board[$idx]=SUNVISITED 
ne 


print_board () 


{ 
Tocai seb 
echo " " 
oie sey sim 17.50 # Reverse order of rows 
do #+ so it prints in chessboard order. 
let "rownum = $row + 1" # Start numbering rows at 1. 
echo -n "Srownum |" # Mark board edge with border and 
itone exolboum, abe (0. 7} #+ "algebraic notation." 
do 
let "idx = SROWS*Srow + $column" 
wie || i{loweiecl|| aeb<||}} -66 SXAUNNNVGUSAETUEND) | 
then 
echo -n "SUNVSYM W ## 
else # Mark square with move number. 
josciteuei ROZE "^ WS Hiooenecl [ach Us. acing = Ww 
ita 
done 
acho = =m HAD] x Wo as a losielksjoace, 
echo # -e enables echoing escaped chars. 
done 
echo " y 
echo " a b (e d e Hs g ws 
} 
failure () 
(ge Waimes, ejes. banil Outs 
echo 
print board 
echo 
echo " Waah!!! Ran out of squares to move to!" 
echo -n " Knight's Tour attempt ended" 
echo " on $(to algebraic $currpos) [square #S$currpos]" 
echo " after just $movenum moves!" 
echo 
exit SFAIL 
} 
zlar Cooircls () # Translate x/y coordinates to board position 
{ #+ (board-array element #). 
# For user input of starting board position as x/y coords. 
# This function not used in initial release of ktour.sh. 
# May be used in an updated version, for compatibility with 
#+ standard implementation of the Knight's Tour in C, Python, etc. 
see |b teas Osa Sey Spa US | 
then 
return SFAIL 


ial 


139 


140 local xc-$1 

i41 local yc=$2 

142 

143 let “board index = Sxe * SROWS + yo" 

144 

145 aie | cboas bmiez -lr SNIN =e) Sboarc iaces ce SMA | 

146 then 

147 return SFAIL # Strayed off the board! 

148 else 

149 return $board index 

150 i3 

Sil p 

1572 

115.3) 

154 

155. wo eileanan (0) # Translate board position (board-array element 4) 
156 1 #+ to standard algebraic notation used by chess players. 
11577 auae d ow WSL] 

158 then 

159) return SFAIL 

160 Rak 

161 

162 local element no-$1 # Numerical board position. 

163 local col ssueve=(( e 19) @ (ol «ex se qp im) 

164 ^aocslrow err) nuc S AL BOG FB: +} 

165 

166 let "zy ao = Seleusuu o / Sirois” 

167 let "col no = $element no % SROWS" 

168 tl-$(col arr[col no]); t2=${row_arr[row_no] } 

169 local apos-S$tl1$t2 # Concatenate. 

170 echo Sapos 

JbydL ji 

LZ 

LTS 

174 

LYS. eeo aeaee 1) # Translate standard algebraic chess notation 
176. 4 #+ to numerical board position (board-array element 4). 
1.73 # Or recognize numerical input & return it unchanged. 
178 ade (p ex Sv ] 

Pes then 

180 return SFAIL 

11 iE aL # If no command-line arg, then will default to random start pos. 
182 

WES MGS ax 

184 local ix count-0 

185 local b index # Board index [0-63] 

186 local alpos-"$1" 

187 

188 arow=S{alpos:0:1} 4 position = 0, length = 1 

189 acol-$(alpos:1:1) 

190 

191 ie [D Sense =~ gekies i] di Numerical input? 

1197 then # POSIX char class 

195 due (|. saco == [EE Sseulgeyatess ||] Ti Number followed by a letter? Illegal! 
194 then return SFAIL 

195 edle sis | Seudjsos gu SMA | (Oti Doare? 

196 then return SFAIL 

197 else return Salpos Return digit(s) unchanged 

19 iE aL + if within range. 

19S itu 

200 i3 

201 

202 ii [I Seu -ee; Swany |I Sa@ol ee SEOWS I] 

23 then # Outside of range 1 - 8? 

204 return SFAIL 


NONNNNNN NH 


N 
«o 


iE aL 


iO abe iim @ ly © ele it oy In 


# Decrementing converts to zero-based array. 


do # Convert column letter to column number. 
alae it use ec SEES v] 

then 

break 

ít 
CEREC OLEA) # Find index count. 
done 
(eue) ) 
let "b index = $ix count + $acol * SROWS" 
if [ $b index -gt $MAX ] 3 Off board? 
then 

return $FAIL 
i3 


isexEwuOa Slo auge» 


generate moves () # Calculate all valid knight moves, 
{ #+ relative to current position ($1), 
#+ and store in $(moves) array. 
local kt hop-1 # One square short leg of knight move. 
local kt_skip=2 # Two squares long leg of knight move. 
local valmov=0 # Valid moves. 
Weyer mow Dosh ler Wro DoS = Sil 3 SCOLSY 
let "movel = -$kt skip + SROWS" # 2 sideways to-the-left, 1 up 
ait [|| "ewe Srov DoS = Sige skip lhe SNIN J] # An ugly, ugly kludge! 
then # Can't move off board. 
move 1=SBADMOVE # Not even temporarily. 
else 
((valmov+t+) ) 
if a 
let "move2 = Ske hop + Skt_skip * SROWS" # 1 sideways to-the-left, 2 up 
ai [| "ese Sro pos = Sk hop ~le SMIN JT # Kludge continued 
then 
move2=$BADMOVE 
else 
( (valmov++)) 
Ea 
let "move3 = Skt hop + $kt skip * SROWS" # 1 sideways to-the-right, 2 up 
ie II epr Sisco joes «e Ghie hoo -Ge SCOTS T 
then 
move3=$BADMOVE 
else 
((valmov+t+) ) 
iE. 
let "move4 = Skt_skip + SROWS" # 2 sideways to-the-right, 1 up 
xi [|| "ege Sacow pos a Skre skio -ce SCOMS Ti 
then 
move 4=SBADMOVE 
else 
((valmov+t+) ) 
ak 
let "move5 = Ski skip - SROWS" # 2 sideways to-the-right, 1 dn 
ai [|| "ewe Sron 90S T Ske skio <E SCOUS I] 
then 
move5=$BADMOVE 
else 


C9 CO CO CO CO CO CO CO 


(9). (9) 
ND | 
C Ko) 


((valmov+t+) ) 
fai 
let "move6 = Ske hop - Skr skip * SROWS" # 1 sideways to-the-right, 
if [| expr Srow pos + Ske hga -ge SCOLS |] 
then 
move 6=$BADMOVE 
else 
((valmov+t+) ) 
stabil 
let "move7 = -Skre hoo =~ Skt_skip = SROWS" # 1 sideways to-the-left, 
if [| expr Srow pos — Skt hop -le SMIN JI 
then 
move 7=$BADMOVE 
else 
((valmov+t+) ) 
ít 
let "move8 = -$kt skip - S$ROWS" # 2 sideways to-the-left, 
aie [|| “exon Sron pos = Su skip =le SNIN I] 
then 
move8=$BADMOVE 
else 
( (valmov++)) 
I3 # There must be a better way to do this. 


2 


2 


i 


dn 


dn 


dn 


local m=( $valmov $movel $move2 $move3 $move4 $move5 $move6 $move7 $move8 


# S{moves[0]} = number of valid moves. 
# S{moves[1]} ... ${moves[8]} = possible moves. 
Chom smile # Elements of array to stdout for capture in a var. 
} 
is on board () # Is position actually on the board? 
( 
3LAE [ [ SIME -lt SMI | | rey =E " SMAX" ] ] 
then 
return SFAILURE 
else 
return SSUCCESS 
teak 
} 
do_move () # Move the knight! 


{ 


local valid_moves=0 
local aapos 
currposl-"$1" 
lmin-$ROWS 

iex-0 

squarel= 

mpm- 

mov- 

declare -a p moves 


HHEHEREERE ETH TEE EEE EHH DECIDE-MOVE ## itt tit tat tH HH at HH HH HE HE HE RENE 
ise || SerarntooS qe SEULS. | 
then # CRITPOS = square #37 
decide_move 
else # Needs a special patch for startpos=37 !!! 
decide_move_patched # Why this particular move and no other ??? 
ial 
Hae HH HEE E TE EH TE FE HE EEE FE HE EE FE HE E HE EEE RE EE EE EE EEE E TE EE HE EE EEE E E EE E E E HE EEE 


) 


ie 3 (( ++movenum )) # Increment move count. 

338 let "square = $currposl + ${moves[iex]}" 

339 

340 TESUHEREHEUEE EHE E E ERE DEBUG HEHEHE HHH HHH 

341 SLE "SDEBUG" ] 

342 then debug 4 Echo debugging information. 

3A 3 ER 

344 HHEPFHEEEEHEREEEEEEEEREE ERE ERE ERE ERE ERE E EE E EE E HE HHH HE HH 

S45 

346 wit [| VSecwairce! e SIMA |I VEgeeue" =ke SKIN || | 

347 $(board[square]) -ne SUNVISITED ] 

348 then 

349 (( --movenum )) # Decrement move count, 
350 echo "RAN OUT OF SQUARES!!!" #+ since previous one was invalid. 
SSL return SFAIL 

952 i3 

3159) 

354 board[square]-$movenum 

355 currpos-$square Update current position. 

356 ((valid_moves++)); moves [0]=$valid_moves 

357 aapos-$ (to algebraic $square) 

359 echo -n "Saapos " 

359 test $(( $Moves $ SLINELEN )) -eq 0 && echo 

360 # Print LINELEN-21 moves per line. A valid tour shows 3 complete lines. 
SGA return Svalid_moves Found a square to move to! 

362. } 

363 

364 

SIG 

366 do move stupid() # Dingbat algorithm, 

S67 4 #+ courtesy of script author, *not* Warnsdorff. 
368 local valid moves-0 

9 local movloc 

370 local squareloc 


3 pil local aapos 
B72 local cposloc-"$1" 


ans 

Sq a oie unowlloe alm 414548] 

S75 do # Move to first-found unvisited square. 

376 let "squareloc = Scposloc + S${moves[movloc] }" 

ST is on board $squareloc 

378 if [ $? -eq $SUCCESS | && [ ${board[squareloc]} -eq S$UNVISITED ] 
97$, then # Add conditions to above if-test to improve algorithm. 
380 (( ++movenum )) 

Ser board[squareloc]=Smovenum 

382 currpos=Ssquareloc # Update current position. 

383 ((valid_moves++) ); # moves[0]-$valid moves 

384 aapos-$ (to algebraic $squareloc) 

385 echo -n "$aapos " 

386 test $(( $Moves $ SLINELEN )) -eq 0 && echo # Print 21 moves/line. 
SU return $valid moves # Found a square to move to! 

388 igi. 

399 done 

390 

39 return SFAIL 

997 # If no square found in all 8 loop iterations, 

393 #+ then Knight's Tour attempt ends in failure. 

394 

395 # Dingbat algorithm will typically fail after about 30 - 40 moves, 
396 #+ but executes much faster than Warnsdorff's in do move() function. 
IJI } 

398 

399 

400 

401 decide move () # Which move will we make? 


402 { # But, fails on startpos-37 !!! 


[E WES Wis Eso ges dus ES ies gES qus ey Qs des qfes quy Q5. des des pes ges des. Wes qiEy quss des ges dE. Jew des dus quy ges des ges gdey Qesy ges. des quy gus. des des des qus. ges. Wes qux qfme. pes des Es qiu des dus qus qj qs qímy gums qe ques. dus ges ges. dés. Wes 


03 


ittowe MOV shin Lo 9] 


do 
let "squarel = Scurrposl + S$(moves[mov])" 
is on board $squarel 
if [I $? -eq $SUCCESS && ${board[squarel]} -eq SUNVISITED ]] 
then Find accessible square with least possible future moves. 
This is Warnsdorff's algorithm. 
What happens is that the knight wanders toward the outer edge 
+ of the board, then pretty much spirals inward. 
Given two or more possible moves with same value of 
+ least-possible-future-moves, this implementation chooses 
+ the first of those moves. 
This means that there is not necessarily a unique solution 
+ for any given starting position. 
possible moves $squarel 
mpm-$? 
p. moves [mov] -$mpm 
if | $mpm -lt Slmin | # If less than previous minimum 
then 4 S 
lmin-$mpm # Update minimum. 
iex=Smov # Save index. 
ita 
FL 
done 
} 
decide_move_patched () # Decide which move to make, 
(oo dX AE #+ but only if startpos-37 !!! 
pone Tue alia. ides oi) 
do 
let "squarel = Scurrposl + S$(moves[mov])" 
is on board $squarel 
if [[ $2 -eq $SUCCESS && ${board[squarel]} -eq S$UNVISITED ]] 
then 
possible moves Ssquarel 
mpm-$? 
p. moves [mov] -$mpm 
si [| Smom -le Silmain |) — e Wie less=-than-0r Soil rao Drava malian! 
then 4 s 
lmin-$mpm 
iex-$mov 
aL 
13 
done # There has to be a better way to do this. 
} 
possible_moves () # Calculate number of possible moves, 
{ #+ given the current position. 
aise [Di og Wis) 7] 
then 
return SFAIL 
ie aL 
local curr_pos=$1 
local valid_mov1l=0 


local icx=0 


ti 
iw) 


[square #Scurrpos]. 


469 local movl 

470 local sq 

471 declar a movesloc 

472 

473 movesloc=( $(generate_moves $curr pos) ) 

474 

475 inoue Well bor {ils ish}! 

476 do 

E let "sq = $curr pos + $(movesloc[movl])" 

478 is. on board $sq 

479 if [ $? -eq $SUCCESS ] && [ ${board[sq]} -eq SUNVISITI 
480 then 

481 (ead MOViE), 

482 Ts 

483 done 

484 

485 return $valid movl # Found a square to move to! 
486 } 

487 

488 

489 strategy () 

490 ( 

491 echo 

492 

493 fae (| in WSS i 

494 then 

495 for Moves in {1..63} 

496 do 

497 cposl-$1 

498 moves-( $(generate moves $currpos) ) 

499 do move stupid "$currpos" 

500 se (| SP ep SEADE | 

501 then 

502 failure 

503 fi 

504 done 

50S iE aL 

506 

507 # Don't need an "else" clause here, 

508 #+ because Stupid Strategy will always fail and exit! 
509 for Moves in {1..63} 

510 do 

Sdri; cposl=$1 

SEZ moves-( $(generate moves Scurrpos) ) 

NM do move "Scurrpos" 

Sl 4l ai [ $9 seer Smau | 

5L5 then 

SLE failure 

SL 7) i3 

Sd 

51.9) done 

520) # Could have condensed above two do-loops into a single one, 
HZ Ail echo #+ but this would have slowed execution. 
522 

BAS) print_board 

524 echo 

525 echo "Knight's Tour ends on $(to algebraic $currpos) 
526 return SSUCCESS 

Sud ge ct 

52 

529 debug () 

S9. 4 # Enable this by setting DEBUG-1 near beginning of script. 
SBi dKeyeretull ia 

5S2 

BSS) cho w 

534 echo " At move number  $movenum:" 


(Gn) Gn AII Cal £n]. n] Gal Gn Gy n] 
AS WES Sy RS SS SMS SESS 
(ee) Sey Gl des. GM d 


[91] 
Oo 
(Sy Wo} 


echo " *** possible moves = Smpm ***" 
# echo "### square = Ssquare ###" 


echo "lmin = Slmin" 
echo "S{moves[@]}" 
FOr am alin {ib oS} 
do 
Scha =m | (Sin) eS {jo moveslal jb * 
done 
echo 
giao Viez = Sue» 33 iene [fue] = SAinewas || ex] JE 
echo "square = $square" 
cho "n "n 
echo 


) # Gives pretty complete status after ea. move. 
# 
#2 oum manm O [f 


rrom algebrai WS iY 


startpos-$? 
xe [T “Seeareaos” see, Viso T f Okay even if no $1. 
then d AO AAS SS Okay even if input 
echo "No starting square specified (or illegal input) ." 
let "startpos = SRANDOM % SSQUARES" # 0- 63 permissable 
ii 
SLE VS2 = Weuewyesbehu | 
then 
STUPID=1 
eyellayoy iol 4 ### Stupid Strategy ###" 
else 
STUPID-'' 
ECNO im M! aw Wüeusimsyolossitit us; vlhsyowcdbieJawg el! 
feit 


initialize board 


movenum-0 


board[startpos]-$movenum 4 Mark each board square with move number. 


che (9 


range. 


currpos-$startpos 
algpos-$ (to algebraic $startpos) 
echo; echo "Starting from $algpos [square 4$startpos] ..."; echo 
echo -n "Moves:" 
strategy "Scurrpos" 
echo 
exit 0 # return 0; 
) 4 End of main() pseudo-function. 
Exercises: 


1) Extend this example to a 10 x 10 board or larger. 
2) Improve the "stupid strategy" by modifying the 
do move stupid function. 


601 Hint: Prevent straying into corner squares in early moves 

602 (the exact opposite of Warnsdorff's algorithm!). 

603 3) This script could stand considerable improvement and 

604 Streamlining, especially in the poorly-written 

605 generate moves() function 

606 and in the DECIDE-MOVE patch in the do move() function. 

607 Must figure out why standard algorithm fails for startpos-37 

608 #+ but not on any other, including symmetrical startpos-26. 

609 Possibly, when calculating possible moves, counts the move back 

610 #+ to the originating square. If so, it might be a relatively easy fix. 


Example A-46. Magic Squares 


! /bin/bash 
msquare.sh 
Magic Square generator (odd-order squares only!) 


Author: mendel cooper 

reldate: 19 Jan. 2009 

License: Public Domain 

A C-program by Kwon Young Shin inspired this script. 
See http://user.chollian.net/-brainstm/MagicSquare.htm 


Definition: A "magic square" is a two-dimensional array 
of integers in which all the rows, columns, 
and *long* diagonals add up to the same number. 
Being "square," the array has the same number 
of rows and columns. 

An example of a magic square of order 3 is: 


Mop pop pp PPP PY 
O (o 00 -1 O Oi i$ (). M. IB. O «o 0 «1 O O1 4S CQ I ES 


e. X 6 

ON COME 

4 9 2 

All the rows, columns, and long diagonals add up to 15. 

21 
22 
2:5) Globals 
24 EVEN-2 
25 MAXSIZE-31 Ww Sil wows x Sil wells. 
26 E_usage=90 # Invocation error. 
27 dimension= 
28 declare -i square 


usage_message () 
{ 
echo "Usage: $0 square-size" 
echo " ... where \"square-size\" is an ODD integer" 
echo " abiol AE laVS) aetehove fey b = Sb 
# Actually works for squares up to order 159, 
#+ but large squares will not display pretty-printed in a term window. 
# Try increasing MAXSIZE, above. 
exit $E usage 


CO CO CO CO CO CO CO CO CO CO Dd 
Wey Keer X] (xy Gal ges 63 IS) |) e» wo 


40 

41. 

412. (xeu (0) # Here's where the actual work gets done. 

43 ( 

44 local row col index dimadj j k cell val-1 

45 dimension-$1 

46 

47 ee Weliinvacls = Sclbumeinsiioim e SUP ler ebony (= 2 fp ox oS, then jerewuove~areey 
48 

49 for ((j=0; J < dimension; j-*-)) 


S for ((k=0; k < dimension; k++) ) 

52 do # Calculate indices, then convert to l-dim. array index. 
53 # Bash doesn't support multidimensional arrays. Pity. 

54 len veol = 9k S3 a Suobbnerlg p. let Wol B= Sielinnerisaroio! 

55 lee "sey = Sq = 2 = S T Suebbnmewusshoi" mg ler Vroon Se SXxelluuexowsia oul 
56 let "index = $row*($dimension) + $col" 

57 square [Sindex]=cell_val; ((cell val-*-*)) 

58 done 

I9 done 

60 } # Plain math, no visualization required. 

61 

62 

63 print square () # Output square, one row at a time. 
64 ( 

65 loeall row «exul ich chil 

66 let "dl = $dimension TA # Adjust for zero-indexed array. 
67 

68 for row in $(seq 0 S$dl) 

69 do 

70 

WAL foie (xol im S (see; (0 Schil) 

UZ do 

13) let "idx = Srow * Sdimension + $col" 

74 joreitioes® VS sich Y WS oeaan ek ep eel san " 

7/8) done # Displays up to 13-order neatly in 80-column term window. 
76 

77) echo # Newline after each row. 

78 done 

TA y 

80 

81 

82 JdESSHETASHEESHSEATHETAHETASHERASHEOSUHETASHETAHETASUHE T E HF 

Ss ase [ps Ysi II Jj JI USI3W gu SABOSSJXZE |] 

84 then 

85 usage message 

86 fi 

87 


88 let "test even = $1 % SEVEN" 
89 if [ S$test even -eq 0 ] 


90 then # Can't handle even-order squares. 

9l usage message 

92. t3 

93 

94 calculate $1 

95 print square # echo "S{square[@]}" # DEBUG 

96 

9] e»t SP 

98 HHH HE HEH HE HE EEE HE HH HE HE EEE E E HE E E E E E E E E EE HE HEE 

99 
100 
LO Exercises: 
102 1 ——————-——— 
103 1) Add a function to calculate the sum of each row, column, 
104 and *long* diagonal. The sums must match. 
LOS armas; ie the Vane ae COMSIEGIMEY OIE cime. jeeucicuC@ulaw Oren Severe, 
106 2) Have the print_square function auto-calculate how much space 
107 to allot between square elements for optimized display. 
108 This might require parameterizing the "printf" line. 
109 3) Add appropriate functions for generating magic squares 
110 with an *even* number of rows/columns. 
CLI TaLe LS non=riiivyia l(i, 
AEZ See the URL for Kwon Young Shin, above, for help. 


Example A-47. Fifteen Puzzle 


1 #!/bin/bash 
2 fifteen.sh 
3 
4 Classic "Fifteen Puzzle" 
5 Author: Antonio Macchi 
6 Lightly edited and commented by ABS Guide author. 
7 Used in ABS Guide with permission. (Thanks!) 
8 
9 The invention of the Fifteen Puzzle is attributed to either 
10 #+ Sam Loyd or Noyes Palmer Chapman. 
al The puzzle was wildly popular in the late 19th-century. 
LZ 
L3 Object: Rearrange the numbers so they read in order, 
JLdb ear cereo. db o Dg 
L5 | il 2 3 4 | 
16 | 5 6 7 Su 
17 | 9 do 3- 322 | 
18 late a. uet) | 
19 
20 
2 
PPEEIIIIIIZIZIIZIZIIXZIXIitii 
28 Constants # 
24 SQUARES=16 # 
25 FAIL=70 # 
26 E PREMATURE EXIT-80 # 
27 HHEEEEEERE EE HEE HE EEE EE HF 
28 
29 
30 ###HEEEH 
Sil Data # 
32 EEHEEHE 
33 
SA Pinz Ze) (melee Seed eos (oy qp ep Om O mesa lene oN eA ele Se eub UE 
35) 
36 
37 FEET EEE EHH 
38 Functions # 
39 FEEEEEE EEE HHH 
40 
41 function swap 
Le 
43 local tmp 
44 


45 tmp-$(Puzzle[$1]) 
46 Puzzle[S$1]-2$(Puzzle[$2]) 
47 Puzzle[$2]=Stmp 


51 function Jumble 
52 { # Scramble the pieces at beginning of round. 
53 local i posl pos2 


54 

55 for 3t aum (il, s 100) 

56 do 

5/7) posl=S(( SRANDOM $ SSQUARES) ) 
58 pos2=S(( SRANDOM $ SSQUARES )) 
59 swap $posl $pos2 

60 done 

Gil. 4. 

62 

63 


64 function PrintPuzzle 


OS: 1 


losca nil 12 jowZoos 
puzpos=0 
clear 
echo "Enter quit to exit."; echo # Better that than Ctl-C. 
chorn a . 5 Ju 4 Top border. 
icone 3bdb be Abs A 
do 
itowe 302; atin fk, a} 
do 
jexcabewesg “|| fuz ws Weeds epo] }} ^ 
(( puzpost* )) 
done 
eh oam 4 Right-side border. 
test $i1 = 4 || echo "+ + + T +" 
done 
eim) Ww ; H y uy 4 Bottom border. 


87 function GetNum 
Se 4 i ues ie wedaol ausus 


iocari 


while 
do 

ec 

rea 

at 

Les 

TSS 
done 

retur 


puznum garbage 


CAUS 


ho "Moves: $moves" # Also counts invalid moves. 

d -p "Number to move: " puznum garbage 

E [| VSopzmumi = Werre JP tren chop Sie dim RNM LILI EL 
t -z "Spuznum" -o -n "${puznum//[0-9]/}" && continue 

t Spuznum -gt 0 -a $puznum -le $SQUARES && break 


n $puznum 


103 function GetPosFromNum 
104 ( 4 $1 = puzzle-number 


105 local puzpos 

106 

MON EO PUZSOS: am (055.15) 

108 do 

109 test "S$(Puzzle[$puzpos])" = "$1" && break 

110 done 

aa return Spuzpos 

qu AE 

LES 

LIA 

115 function Move 

116 ( 4 $1=Puzzle-pos 

Ly teste Gil = 3 ce tes: USeweehelete 9-394] 9 —w wy 
118 && swap S1 $(( $1 - 4 )) && return 0 

119 ese SAN Sis 9) sexe S. dr SEE VS Tier dics ES UTC Sal se SE YE TERM. ex ur cu 
1.2/0) && swap S1 $(( S1 + 1)) && return 0 

d ros. Sil Hike iA we cess WSS Tet Sal se «t 9) [3E ese UE 
122 && swap $1 $(( $1 + 4 )) && return 0 

12:5 este SC Sul 91) ane: 0) Gite Eel aei pZvzd b le UC Bul es db ope SE UE fa 
124 svap Si S(¢ Si = 1 y) Ge return 0 

TAS Tee. di 

WAG 3 

LX d 

128 


129 function Solved 


139 1 


JST local pos 


JUS 

133 Or OS 3um {0.14} 

134 do 

135 test "Sirumzile(Spos]}" = S«( Sos + 1 )) || recura SEALL 
136 # Check whether number in each square = square number. 
TSI done 

138 return 0 4 Successful solution. 

139 

140 

141 


142 #HHTHHHEEEEEPPEEEEE MAIN () ft eae a HH HHH HE EEE SH HT 
143 moves=0 


144 Jumble 

145 

146 while true # Loop continuously until puzzle solved. 
147 do 

148 echo; echo 

149 PrintPuzzle 

150 echo 

Syl while true 

152 do 

1153) GetNum 

154 puznum-$? 

1515 GetPosFromNum $puznum 
156 puzpos-$? 

MS ( (moves-t-)) 

115 Move Spuzpos && break 
159) done 

160 Solved && break 

161 done 

162 


163 echo;echo 

164 PrintPuzzle 

165 echo; echo "BRAVO!"; echo 

166 

167 exit 0 

168 FHFFEHEFEPEEEPEEEPEEEEE ERE EEE EEE HEHE HEHE HEHE HEHE F 
169 
170 # Exercise: 

iL «o ———————— 

172 # Rewrite the script to display the letters A - O, 
173 #+ rather than the numbers 1 - 15. 


Example A-48. The Towers of Hanoi, graphic version 


1 4! /bin/bash 

2 The Towers Of Hanoi 

3 Original script (hanoi.bash) copyright (C) 2000 Amit Singh. 
4 All Rights Reserved. 

E http://hanoi.kernelthread.com 

6 

3i hanoi2.bash 

8 Version 2.00: modded for ASCII-graphic display. 

9 Version 2.01: fixed no command-line param bug. 
10 Uses code contributed by Antonio Macchi, 
11 #+ with heavy editing by ABS Guide author. 
12 This variant falls under the original copyright, see above. 
LS Used in ABS Guide with Amit Singh's permission (thanks!). 
14 
15 
16 ### Variables && sanity check ### 


17 


18 E NOPARAM-86 

19 E BADPARAM-87 # Illegal no. of disks passed to script. 
20 E NOEXIT-88 

21. 


22 DISKS-$(1:-E NOPARAM) # Must specify how many disks. 
23 Moves=0 

24 
25 MWIDTH-7 

26 MARGIN-2 

27 # Arbitrary "magic" constants; work okay for relatively small # of disks. 
28 4 BASEWIDTH-51 # Original code. 


29 let "basewidth = SMWIDTH * SDISKS + SMARGIN" # "Base" beneath rods. 
30 # Above "algorithm" could likely stand improvement. 
Sil 


32 ### Display variables Tg 
33 ler Vchisieel = SSH = 1 

34 let "spacesl = SDISKS" 

35 let "egere = 2 = SIDIUSEKGS 


36 

37 lee "leaowe ic = SDISES = 1" # Final move? 
38 

39 

40 declare -a Rodl Rod2 Rod3 

41 

42 ### FE FE TE FE TE FE FE FE FE FE FE FE FE EEE HEHE HE HE HE HE HF ### 

43 

44 

45 function repeat { # $l=char $2=number of repetitions 
46 LOCUL aei # Repeat-print a character. 
47 

48 ite (( m=O meS2e irk 2959 clo 

49 echo =n "si" 

50 done 

od j 

52 

53 function FromRod A 

54 local rod summit weight sequenc 

55 

56 while true; do 

57 rod=$1 

58 Ease Spseed/l^in22])41 T] eemidswe 

59 

60 sequence=S (echo $(seq 0 $disksl | tac)) 

61 for summit in $sequence; do 

62 eval weight=\${Rod${rod} [$summit]]) 

63 test Sweight -ne 0 && 

64 ( echo "$rod $summit Sweight"; return; ) 
65 done 

66 done 

67 ] 

68 

69 


70 function ToRod { # $1-previous (FromRod) weight 
3 logell rocl iE aie sie ae 1c weight sequenc 


Ce 

p while true; do 

74 3c'oyole S 2. 

75 Test (S fixe [Ol23]} || || come sae 

76 

EN sequence=S (echo $(seq 0 $disksl | tac)) 

78 WONe Isbiesie i ie in Ssequence; do 

79 eval weight=\${Rod${rod} [$firstfree] } 

80 test Sweight -gt 0 && { (( firstfreet+t+ )); break; } 
81 done 


82 test $weight -gt $1 -o $firstfree = 0 && 


84 


88 
89 


pot 
«o 


fu 


di 
{ 


ee 
SIE 
ilm 


# 


do 


| echo Yro Siicswixees seSieiwiueing 
done 


nection PrintRods { 
local disk rod empty fill sp sequence 


repeat " " Sspacesl 

exeo) -—m |) 

repeat " " $spaces2 

EINO aay WW 

repeat " " $spaces2 

echo "[* 

sequence-$ (echo $(seq O $disksl | tac)) 


for disk in $sequence; do 
one aetoxol in Hil, o Shs clo 


} 


eval empty-$(( S$DISKS - (Rod${rod}[$disk] / 2) )) 
eval fi11=\${Rod${rod} [$disk] } 
repeat " " Sempty 
ESSE pEaLILIL cene 0) 4 repeat "Uus esti echon =a. ww 
repeat " " Sempty 
done 
echo 
done 
repeat "=" $basewidth # Print "base" beneath rods. 
echo 
splay () 
echo 
PrintRods 


# Get rod-number, summit and weight 
mies ( ~ imicemikyere! Sd ) 
eval Rod${first[0]}[${first[1]}]=0 


# Get rod-number and first-free position 
Seconc=( "eme Sciseew[21] S92" ) 


eval Rod${second[0]}[${second[1]}]=${first [2] } 


ho; echo; echo 
[ "$(Rod3[lastmove t])" = 1 ] 


en # Last move? If yes, then display final position. 


echo "+ Final Position: $Moves moves"; 
PrintRods 
i3 


echo 


From here down, almost the same as original (hanoi.bash) 


hanoi() { # Recursive function. 
case $1 in 
0) 
x) ? a 
donano WS(qe1-1yyv.$2 $4 $35 
a | "USWiwes" -ne © ] 


then 
echo "+ Position after move $Moves" 


SGI USE c 


149 ita 


quo ((Moves-t-)) 

LSI Scng =m Y Next move will be: W 
152 echo $2 "--»" $3 

1153 display $2 $3 

154 Gohamon US((S1-1))w S4 $3 82 
155 Pi 

156 esac 

ilu 

158 

159 

160 setup_arrays () 

d 4 

162 local dim n elem 

163 


164 leg vonmi = Su = d 
165 elem=Sdiml 


166 

167 fOr m im Siiseer 0 Selimi) 
168 do 

169 let "Rodl[$elem] = 2 * $n + 1" 
170 Rod2 [$n]=0 

PAL Rod3 [$n]=0 

172 ( (elem--)) 

LIS done 

17a p 

S 

176 

177 did Main T 

LPS 


179 setup arrays SDISKS 

180 echo; echo "+ Start Position" 

181 

182 case $4 in 

183 1) case $(($1>0)) in # Must have at least on 


184 1) 
195) disks-$1 
186 Colaaimes Sil i 9 2 


187 # Total moves Zum 1, where n = number of disks. 


188 echo 

189 exzalic (Os 
190 Pi 

191 *) 


1192 echo "$0: Illegal value for number of disks"; 


193 exit $E_BADPARAM; 
194 Pi 

dS esac 

LOG 
197 
iL Sg} clear 

199 echo "usage: $0 N" 


x € 
- 


200 echo " Where \"N\" is the number of disks." 


201 exit SE_NOPARAM; 

202 ae 

203 esac 

204 

205 Exe, Sis INOMPKICI i? iSuexexbulolo 1. (E. devenue 
206 

207 4 Note: 


208 # Redirect script output to a file, otherwise it scrolls off display. 


Example A-49. The Towers of Hanoi, alternate graphic version 


NPRPRPPRP PPP PY 
SCOMIDTDHRWNHHFOWOAIDRGUBRWNE 


WWNNNNNNDN DN NH 
[= (€» iwe) er =a) ony Gal qm ee ISS) [SS 


WWWW CO CO CO CO 
We) Gor on) Gal des (3. [55 


A e 
T 


fob bP PB am us 
Wey Kesh J] (e»y (On des (69. [5S 


U O1 
e o 


(On. Gal ion (Oni (On) (mj (mi (Oni 
(o) Keo) i) copy (Gal HSS (69) 55) 


(ny Xen Ty CR (63, Lenk 19») 
(ox Gi de Gs) IS) [oS 


! /bin/bash 
he Towers Of Hanoi 


T 

O 

All Rights Reserved. 
http://hanoi.kernelthread.com 


hanoi2.bash 

Version 2: modded for ASCII-graphic display. 
Uses code contributed by Antonio Macchi, 

+ with heavy editing by ABS Guide author. 


Used in ABS Guide with Amit Singh's permission 


Variables # 
NOPARAM=8 6 


E NOEXIT-88 


UJ 


DISKS-$1 
oves-0 


WIDTH-7 
ARGIN=2 


This variant also falls under the original copyright, 


riginal script (hanoi.bash) copyright (C) 2000 Amit Singh. 


(thanks!). 


BADPARAM=8 7 # Illegal no. of disks passed to script. 


ny # Interval, in seconds, between moves. Change, 


see above. 


if desired. 


# Arbitrary "magic" constants, work okay for relatively small # of disks. 


# BASEWIDTH-51 # Original code. 


let "basewidth = SMWIDTH * SDISKS + $MARGIN" # "Base" 


# Above "algorithm" could likely stand improvement. 


# Display variables. 
let "disksl = $DISKS - 1" 
let "spacesl = SDISKS" 


let "spaces2 = 2 * SDISKS" 
let VWlasicmewea ic = SDILSIKS = IW # Final move? 
declare -a Rodl Rod2 Rod3 
HEHEHE HEHEHE EE HEH HE 
function repeat { # S$l=char $2=number of repetitions 
lkexeell. igi # Repeat-print a character. 
ieu (( m=O0p iSc uu Ye Glo 
echo =m WSL 
done 
} 


function FromRod ( 
local rod summit weight sequenc 


while true; do 


rod-$1 
ESSE Siesd/l^1231]41 |||) Conen 
sequence-$ (echo $(seq 0 $disksl | tac)) 


for summit in $sequence; do 
eval weight=\${Rod${rod} [S$summit] } 
test Sweight -ne 0 && 
( echo "Srod $summit $weight"; return; 


done 
done 


beneath rods. 


68 
69 


3/3 
72 


82 


86 


[n 
«o 


function ToRod 


eyes 3etoxol ERESSE E 


while true; d 
rod-$2 


{ 


O 


# $l=previous (FromRod) weight 


wes fef xexe LAIZ i conti nve 


sequence=$ (echo $(seq 0 $disks1 


POCE S ERE 


in $sequence; do 


weight sequenc 


| tac)) 


eval weight=\${Rod${rod} [$firstfree] } 
test Sweight -gt 0 && { (( firstfreet+t+ )); break; } 


done 


test Sweight 


{ echo 
done 


gt $1 -o Sfirstfree = 0 && 


Siceel Srirgrrr ros} 


HUMCELOM Peme Rode 1 
local disk rod empty fill sp sequence 


cote Cujo 5. (9 


repeat " " Sspacesl 
ceho- m ang 

repeat " " $spaces2 
echo m ya 

repeat " " $spaces2 
eoo vM 


sequence-$ (echo $(seq 0 $disksl 
for disk in $sequence; do 
oie sexe! sum, {il,,3hs lo 
eval empty-$(( SDISKS - (Rod${rod}[$disk] / 2) ) 
eval £111=\${Rod$ {rod} [$disk] } 


repeat " " Sempty 
test $fill -gt 0 && repeat 
repeat " " Sempty 
done 
echo 
done 
repeat "=" Sbasewidth # Print 
echo 
} 
display () 
{ 
echo 
PrintRods 


# Get rod-number, 


iasesic=( TROS S4 ) 
eval RƏS rirse lol lS rirst [E 1E ]] ]=0 


return; } 


Bac 


Hest Surati || || elo =i 


"base" beneath rods. 


summit and weight 


# Get rod-number and first-free position 
eexeomeEs( "ed Sistiesie [2] } 92 
eval Rod${second[0]}[${second[1]}]=${first [2] } 


ine [p “USirecls | lasicimewSs c)" 3] 


then # Last move? If yes, 


iEjouhe Gio) (0) 
echo; echo 
PrintRods 


0 
ns 


Final Position: 


) 


$Moves moves" 


) 


then display final position. 


133 
134 
LSS) 
136 
137 
138 
159; 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
TS 
152 
153 
154 
1515) 
156 
15/9) 
158 
15$ 
160 
161 
162 
1165.5) 
164 
165 
166 
167 
168 
169 
170 
173 
372 
JL TES) 
174 
1S) 
176 
p, 
178 
LYS 
180 
181 
182 
183 
184 
185 
186 
187 
188 
189 
190 
Lgl 
1192 
193 
194 
195 
196 
197 
198 


iE aL 


Sleep SDELAY 


# From here down, almost the same as original (hanoi.bash) 
dohanoi() { # Recursive function. 

case $1 in 

0) 


Close VES(qSI-n1)yp" S2 Sa S3 
if [ "SMoves" -ne 0 ] 
then 
joule cuig 0 © 
echo; echo "+ Position after move $Moves" 


ít 

((Moves-t-)) 

Seno =i i Next move will be: n 
echo $2 "--»" $3 


display $2 $3 
dohanod3 "S C(ST-I1)y" $4 53 $2 
esac 


setup arrays () 


{ 


local dim n elem 


lee “elim = Sal = dps 
elem=Sdiml 


£on mn ane si(seq O Sidamil) 
do 
lert "Ueewi[Selsw| = 2 5 Sm Liv 
Rod2 [$n]= 
Rod3 [$n]= 
( (elem--) 
done 


0 
0 
) 


HHH Main HEH 
mejo Wigjobie Cmo ©) 
EOUNE. CES 

clear 


setup arrays S$DISKS 


Tawe ens (9 © 
echo; echo "+ Start Position" 


case S# in 


i) Case: S9) aim # Must have at least one disk. 
1) 
disks-$1 
cahangir Si i 3 2 
# Total moves = 2^n 1, where n = # of disks. 
echo 
exit 0; 


mr 


echo "$0: Illegal value for number of disks"; 


SUSIE c 


E BADPARAM; 


[e 
o 
* (0 
OX 
es 
p 
Eo 


200 PE 

201 esac 

202 Eel 

2/03 ) 

204 echo "usage: $0 N" 

205 echo * Where \"N\" is the number of disks." 
206 exit SE NOPARAM; 

207 He 

208 esac 

209 

21.0) ewig Sm, NOBI # Shouldn't exit here. 

2 1 3. 

212 # Exercise: 

QULS. Gp SSS 

214 # There is a minor bug in the script that causes the display of 
215 #+ the next-to-last move to be skipped. 


26 s pss enle a 


Example A-50. An alternate version of the getopt-simple.sh script 


1 #!/bin/bash 

2 4 UseGetOpt.sh 

3 

4 # Author: Peggy Russell <prusselltechgroup@gmail.com> 
3 

6 UseGetOpt () ( 


7 declare inputOptions 
8 declare -r E_OPTERR=85 
9 declare -r ScriptName=${0##*/} 
10 declare -r ShortOpts-"adf:hlt" 
alate declare -r LongOpts="aoption, debug, file:,help,log,test" 
T2 
13 DoSomething () ( 
14 echo "The function name is '${FUNCNAME}'" 
LS # Recall that SFUNCNAME is an internal variable 
16 #+ holding the name of the function it is in. 
17 } 
18 
19 inputOptions-$(getopt -o "S$(ShortOpts)" --long \ 
20 "S{LongOpts}" --name "${ScriptName}" -- "$(Q)") 
21 
22 ae (DE (Se cue ©) | (ex o—— 0) lp iiem 
2.3 echo "Usage: ${ScriptName} [-dhlt] {OPTION...}" 
24 exit $E OPTERR 
25 ipa 
26 
2:7) eval set -- "S{inputOptions}" 
28 
# Only for educational purposes. Can be removed. 
# 
echo "++ Test: Number of arguments: [S$#]" 


echo '++ Test: Looping through "$Q"' 
tow a im YSEVe clo 
echo VY te salt 
done 
# 


while true; do 
caseum 
= AIC || -m) # Argument found. 
echo “Ojerealeim [S1]! 


CO CO CO CO CO CO CO CO CO CO Dd 
© co So) Ce OS | oe © 


A A 
[— €» 


42 
43 
44 
45 
46 
47 
48 
49 
50 
Sl 
52 
55 
54 
59 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
pit 
V2 
43 
74 
15 
76 
pu 
78 
US 
80 
81 
82 
aS 
84 
(915) 
86 
87 
88 
89 
90 
91 
92 
DS 
94 
9S) 
96 
2 
98 
99 
100 
OL 
102 
103 
104 
105 
106 
107 


=—Cleleibie; || e 
echo "Option 


vr 


# 
[$1] 


Enable informational messages. 
Debugging enabled" 


=Sitilke | -—3) # Check for optional argument. 
Case UB sg #+ Double colon is optional argument. 
I # Not there. 
cecho WOprion PS1] wee cleieauike 
shift 
Pr 
*) # Got it 
echo VOjsicikom ISi] Usine imore [92] V 
Slava. iE 
Pr 
esac 
DoSomething 
Pr 
cx-Jbexer || cb) G7 nasle Terent 


echo "Option 


LN 


sisi oM M NET. 
echo "Option 


LA 


cde || n) 
echo "Option 
break 


vr 


==) # Done! 
echo "Option 
break 
ri 
E) 
echo "Major 
exit 8 


LN 


esac 


echo "Number of arguments: 


shift 
done 


shift 


Only for educational purposes. 


[$1] Logging enabled" 


Enable testing. 
[$1] Testing enabled" 


[$1] Display help" 


$4 is argument number for "--" 


[$1] Dash Dash" 


imeenea Erre fl IU 


[$4] " 


Can be removed. 


"++ Test: 
Jae Wet B 
"$0"; do 
ante [Se 


echo 
echo 
fue d aum 
echo " 
done 


Number of arguments after \"--\" is 
Looping through "S@"" 


[$#] They are: 


[eq Y 


F you can uncomment 


TERREA ER ERR RE RE HE RE RESETETSESTESST M A IN FERRER EERE EE HE HE EEE 
If you remove "function UseGetOpt 


©) 4% gusicl woreesinomioline Wi, 


the "exit 0" line below, and invoke this script 


t with the various options from the command-line. 


108 # 


109 # exit 0 

LLO 

Til acing Vuesie db 

112 UseGetOpt -f myfile one "two three" four 
Lis 

114 echo;echo "Test 2" 

115 UseGetOpt -h 

116 

117 echo;echo "Test 3 Shore (OpXEat aus 

118 UseGetOpt -adltf myfile  anotherfile 

11 

120 echo;echo "Test 4 Long Options" 

121 UseGetOpt --aoption debug log test file myfile anotherfil 


123 exit 


Example A-51. The version of the UseGetOpt.sh example used in the Tab Expansion appendix 


1 #!/bin/bash 

2 

3 UseGetOpt-2.sh 

4 Moci fiec version of toe script for illustrating icelo-exoamsiom 
5 #+ of command-line options. 

6 See the "Introduction to Tab Expansion" appendix. 

7 

8 EPoessailele gyitigueg a =C =i =L -€t -m 

9 #+ -—-aoption, debug £31 log test help 
10 
JL Author of original script: Peggy Russell <prusselltechgroup@gmail.com> 
L2 
13 
14 UseGetOpt () ( 
iS) declare inputOptions 
16 declare -r E_OPTERR=85 
17 declare -r ScriptName-S$([(044*/) 
18 declare -r ShortOpts-"adf:hlt" 
JL) declare -r LongOpts-"aoption,debug,file:,help,log,test" 
20 
21 DoSomething () ( 
BE echo "The function name is '${FUNCNAME}'" 
23 } 
24 
25 input Options=S (getopt -o "S{ShortOpts}" —-long \ 
26 "S{LongOpts}" --name "${ScriptName}" -- "S${@}") 
2 
28 aii [I (S2 sae ©) |] (S; ee 0) lip then 
29 echo "Usage: ${ScriptName} [-dhlt] {OPTION...}" 
30 exit $E OPTERR 
Sil fi 
32 
33 eval set —— "S{inputOptions}™ 
34 
35 
36 while true; do 
EY case "S{1}" in 
38 ——aegptnom [| =a) # Argument found. 
39 Sene VOptiom [S1]! 

40 P 

41 

42 = debug || «e # Enable informational messages. 


43 echo "Option [$1] Debugging enabled" 


46 Salle | =i8) # Check for optional argument. 


47 case "$2" in #+ Double colon is optional argument. 


48 UD # Not there. 

49 echo "Option [$1] Use default" 
50 shift 

5i PP 


513) *) 4 Got it 

54 seha VOyowskey ISL] Usine imore ISAY 
E Spale 

56 Pi 

58 esac 


59 DoSomething 
60 ii 


62 --Jhexer || L wp Mineola grep 
63 echo "Option [$1] Logging enabled" 
64 Pr 


66 --test | -t) # Enable testing. 
67 echo "Option [$1] Testing enabled" 
68 Pi 


70 = helpi 80) 

pa echo "Option [$1] Display help" 
UZ break 

1:8) EB 


715) =>) # Done! S# is argument number for "--", $Q is 


76 echo "Option [$1] Dash Dash" 
17 break 
78 BE 


80 x) 

81 echo "Major internal error!" 
82 exit 8 

83 ii 

85 esac 

86 echo "Number of arguments: [S$#]" 
87 shift 

88 done 


90 shift 


94 exit 


Example A-52. Cycling through all the possible color backgrounds 


#!/bin/bash 


show-all-colors.sh 

Displays all 256 possible background colors, using ANSI 
Author: Chetankumar Phulpagare 

Used in ABS Guide with permission. 


(99) sal (ox; (Ob dE Ger IS) d 
Se SH ch ok 


scap 


sequences. 


9 T2=6 
10 T3236 
11 offset=0 
12 
ILS. sexe n atin (O55 7} 
14 do { 
15 oe ine sum {O, 1j 
16 eo i 
allay shownum— echo YWSorftses + Srl Sinum" onuni 5c: 
18 echo -en "\E[0;48;5;${shownum}m color ${shownum} \E[0Om" 
19 } 
20 done 
All echo 
22 } 
23 done 
an 
25 offset=16 
21 Oe a aa Arg t 
BI oy 
26 Por nine ean. dA SU 
29 do { 
30 Lor OPIN Se CE DU cS 
Sil elo 4 
32 shownum= echo "Soffset + ST2 = S${num3} \ 
319 a Suan ae Sues) *w fep [L| exer 
34 echo en "\E[0;48;5;${shownum}m color ${shownum} V 
35 } 
36 done 
Sil echo 
38 } 
39) done 
40 } 
41 done 
42 
43 offset=232 
AA Tur numi nc Dock] 
45 do { 
46 shownum= expr Soffset + S$numl^ 
47 echo -en "\E[0;48;5;${shownum}m $(shownum) NE [0m" 
48 } 
49 done 
50 
51 echo 


To end this section, a review of the basics .. . and more. 


Example A-53. Basics Reviewed 


1 #!/bin/bash 

2 basics-reviewed.bash 

E 

4 Fil xtension == *.bash == specific to Bash 

5 

6 Copyright (c) Michael S. Zick, 2003; All rights reserved. 
7 License: Use in any form, for any purpose. 

8 Revision: SID$ 

£ 
LO laelaliceyel 3t(ue- JLewpowxouE. los MAO 
11 (author of the "Advanced Bash Scripting Guide") 
327 Fixes and updates (04/08) by Cliff Bamford. 
13 
14 
LS This script tested under Bash versions 2.04, 2.05a and 2.05b. 


E [ Om" 


16 # It may not work with earlier versions. 


17 # This demonstration script generates on intentional-- 

18 #+ "command not found" error message. S line 436. 

19 

20 # The current Bash maintainer, Chet Ramey, has fixed the items noted 
21 #+ for later versions of Bash. 

22 

223 

24 

I5 Tg T 

26 ### Pipe the output of this script to 'more' ### 

Dy ###+ else it will scroll off the page. ### 

28 LEE Ta 

DIO ### You may also redirect its output Td 

30 ###+ to a file for examination. HHH 

Sil HHH HHH 

32 

33 

34 

315) Most of the following points are described at length in 

36 #+ the text of the foregoing "Advanced Bash Scripting Guide." 
37 This demonstration script is mostly just a reorganized presentation. 
38 == Ms 

39 


40 Variables are not typed unless otherwise specified. 
41 


42 Variables are named. Names must contain a non-digit. 
43 File descriptor names (as in, for example: 2»&1) 

44 #+ contain ONLY digits. 

45 

46 Parameters and Bash array elements are numbered. 

47 (Parameters are very similar to Bash arrays.) 

48 


49 A variable name may be undefined (null reference). 
50 unset VarNull 


54 

52 A variable name may be defined but empty (null contents). 

53 VarEmpty-'' # Two, adjacent, single quotes. 
54 


55 A variable name may be defined and non-empty. 
56 VarSomething-'Literal' 


57 

58 A variable may contain: 

59 * A whole number as a signed 32-bit (or larger) integer 
60 "^ JA OVILE] 

61 A variable may also be an array. 

62 

63 A string may contain embedded blanks and may be treated 
64 #+ as if it where a function name with optional arguments. 
65 

66 The names of variables and the names of functions 

67 #+ are in different namespaces. 

68 

69 

70 A variable may be defined as a Bash array either explicitly or 
71 #+ implicitly by the syntax of the assignment statement. 

72 IEisxpsliei ette 

73 declare -a ArrayVar 

74 

715 

76 


77 $ The echo command is a builtin. 
78 echo $VarSomething 

79 

80 # The printf command is a builtin. 
81 # Translate $s as: String-Format 


82 
83 


n 
«o 


pr 
ec 


E 


de 


# 
ec 
ec 


# 
# 


# 
ec 
ec 


intf $s $VarSomething # No linebreak specified, none output. 
ho # Default, only linebreak output. 


The Bash parser word breaks on whitespace. 
Whitespace, or the lack of it is significant. 
(This holds true in general; there are, of course, exceptions.) 


Translate the DOLLAR SIGN character as: Content-Of. 


Extended-Syntax way of writing Content-Of: 
ho ${VarSomething} 


The ${ ... ) Extended-Syntax allows more than just the variable 
name to be specified. 
In general, $VarSomething can always be written as: ${VarSomething}. 


Call this script with arguments to see the following in action. 


Outside of double-quotes, the special characters @ and * 
Specify identical behavior. 
May be pronounced as: All-Elements-Of. 


Without specification of a name, they refer to the 
pre-defined parameter Bash-Array. 


Glob-Pattern references 
lavey fee # All parameters to script or function 
imo Gump # Same 


Bash disables filename expansion for Glob-Patterns. 
Only character matching is active. 


All-Elements-Of references 
ho $@ # Same as above 
ho ${@} # Same as above 


Within double-quotes, the behavior of Glob-Pattern references 
depends on the setting of IFS (Input Field Separator). 
Within double-quotes, All-Elements-Of references behave the sam 


Specifying only the name of a variable holding a string refers 
to all elements (characters) of a string. 


To specify an element (character) of a string, 
the Extended-Syntax reference notation (see below) MAY be used. 


Specifying only the name of a Bash array references 


148 
149 
150 
LS 
152 
15:5) 
154 
1:55 
156 
157 
158 
15$; 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
A73 
17/2 
LAS) 
174 
175 
176 
JU) 
178 
LAE 
180 
Tem 
182 
183 
184 
185 
186 
187 
188 
189 
190 
Lil 
192 
193 
194 
195 
196 
1197 
198 
199 
200 
201 
202 
203 
204 
205 
206 
207 
208 
209 
210 
ZUM 
Zale? 
lS) 


+ the subscript zero element, 
+ NOT the FIRST DEFINED nor the FIRST WITH CONTENTS element. 


Additional qualification is needed to reference other elements, 
+ which means that the reference MUST be written in Extended-Syntax. 
The general form is: ${name[subscript]}. 


Ihe string forms may also be used: $(name:subscript) 
+ for Bash-Arrays when referencing the subscript zero element. 


Bash-Arrays are implemented internally as linked lists, 
+ not as a fixed area of storage as in some programming languages. 


Characteristics of Bash arrays (Bash-Arrays): 


If not otherwise specified, Bash-Array subscripts begin with 
+ subscript number zero. Literally: [0] 
This is called zero-based indexing. 


If not otherwise specified, Bash-Arrays are subscript packed 
+ (sequential subscripts without subscript gaps). 


Negative subscripts are not allowed. 


ti 


lements of a Bash-Array need not all be of the same typ 


ti 


lements of a Bash-Array may be undefined (null reference). 
That is, a Bash-Array may be "subscript sparse." 


ti 


lements of a Bash-Array may be defined and empty (null contents). 


ti 


lements of a Bash-Array may contain: 
* A whole number as a signed 32-bit (or larger) integer 
= A it ailing; 
* A string formated so that it appears to be a function name 
+ with optional arguments 
## 
Defined elements of a Bash-Array may be undefined (unset). 
That is, a subscript packed Bash-Array may be changed 
ab into a subscript sparse Bash-Array. 
LE 


Elements may be added to a Bash-Array by defining an element 
* not previously defined. 
## 

For these reasons, I have been calling them "Bash-Arrays". 
I'll return to the generic term "array" from now on. 
== meu 


CRO H 


# Lines 202 - 334 supplied by Cliff Bamford. (Thanks!) 
# Demo --- Interaction with Arrays, quoting, IFS, echo, * and @ cg 
#+ all affect how things work 


ArrayVar[0]-2'zero' 0 normal 
ArrayVar[1]-one 1 unquoted literal 
ArrayVar[2]-2'two' 2 normal 
ArrayVar[3]-2'three' 3 normal 
ArrayVar[4]='I am four' 4 normal with spaces 
ArrayVar[5]-'five' 5 normal 

unset ArrayVar[6] 6 undefined 
ArrayValue[7]='seven' % norma | 


214 
215 
216 
ZIT 
218 
DUG 
220 
2A 
222 
ZL 
224 
225 
226 
2271) 
228 
229, 
230 
23 
232 
233 
234 
2.315 
236 
23 
238 
239 
240 
241 
242 
243 
244 
245 
246 
247 
248 
249 
250 
251 
252 
253 
254 
255 
256 
257 
258 
2/59 
260 
261 
262 
263 
264 
265 
266 
267 
268 
269 
270 
2TH 
272 
Z TES 
274 
Zul 
276 
273 
278 
ZUG 


ArrayValue[8]='' 


# 8 defined but empty 


ArrayValue[9]='nine' # 9 normal 
cho ' Here is the array we are using for this test' 
echo 
echo "ArrayVar[0]-2'zero' |). io eye V 
echo "ArrayVar[1]=one 1 unquoted literal" 
echo "ArrayVar[2]-2'two' 2 ie voneuneui 
echo "ArrayVar[3]='three' d mordet 
echo "ArrayVar[4]="I am four' 4 normal with spaces" 
echo "ArrayVar[5]-'five' S swoscunel 
echo "unset ArrayVar[6] 6 undefined" 
echo "ArrayValue[7]="seven' 7 normal" 
echo "ArrayValue[8]='' 8 defined but empty" 
echo "ArrayValue[9]='nine' 9 ime renner it 
echo 
echo 
echo '---Case0: No double-quotes, Default IFS of space,tab,newlin 
ISHS N20] TSU Nr (0) GU SY VN! # In exactly this order. 
Gyelove) kers ies minei se AS (Mucieehyweue [L5 1] jh! 
printf $q ${ArrayVar[*] } 
echo 
echo 'Here is: printf %q {${ArrayVar[@]}' 
printf $q ${ArrayVar[@] } 
echo 
Clove) Mere L3 Bela SwucieayAvele || 1] JEU 
echo S${ArrayVar[@] } 
cho 'Here is: echo {S${ArrayVar[@]}' 
echo ${ArrayVar[@€] } 
echo 
echo '---Casel: Within double-quotes - Default IFS of space-tab- 
newlines ===" 
IESZSS' x20 S Nx09 ST NsEQA $t Taes im bytes, 
echo 'Here is: printf $q "{${ArrayVar[*]}"' 
jolene Sey "Sere eue [Ls ]] 35 
echo 
Secho "leue: tss joudahineie Sep WS pese Vei LENI y 
printf Sq "${ArrayVar[@]}" 
echo 
cho ‘Here is: echo "S{ArrayVar[*]}"' 
echo "S{ArrayVar[@]}" 
cho 'Here is: echo "($(ArrayVar[G])"' 
echo "S{ArrayVar[@]}" 
echo 
echo '---Case2: Within double-quotes = IFS is q' 
IFS-'q' 
echo 'Here is: printf $q "{S{ArrayVar[*]}"' 
Dete Sep V SUAE sue [Lo ]] 1}! 
echo 
echo kere abs ewe: “xe; "dL ere wea [I8] PY 
printf %q "S${ArrayVar[@]}" 
echo 
cho 'Here is: echo "S{ArrayVar[*]}"' 
echo "S{ArrayVar[@]}" 
cho 'Here is: echo "{S${ArrayVar[@]}"! 
echo "S{ArrayVar[@]}" 
echo 
eleg; Y=—SCASSSs Mira Couleurs = ume Lg ^U 
US 
echo 'Here is: printf $q "{S{ArrayVar[*]}"' 


QU» (69 (V5. (GS) ds (3 (63 ED) 
(69) — (oxi (| de Go. IS) [E 


w w 
N H 
ENNO. 


primer ep VSA ren Vee p» 

echo 

echo 'Here is: printf %q "{${ArrayVar[@]}"' 
printf $q "S${ArrayVar[@]}" 


echo 

cho 'Here is: echo "S{ArrayVar[*]}"' 
echo "S{ArrayVar[@]}" 

cho kere isi echo "JS ze Wiese [0] jv 


echo "S{ArrayVar[@]}" 


ecno 


echo '---Case4: Within double-quotes - IFS is ^ followed by 


space,tab,newline' 

SSS UOUSU NS AOU UN ISIS Y Ns Op 
Gels. Vise Le joni Se UL Anen eue ls | EV 
printf $q "S(ArrayVar[*])" 

echo 
echo 'Here is: printf $q "{${ArrayVar[@]}"' 
printf Sq "${ArrayVar[@]}" 


echo 

cho 'Here is: echo "S{ArrayVar[*]}"' 
echo "S${ArrayVar[@]}" 

cho 'Here is: echo "{${ArrayVar[@]}"! 


echo "S${ArrayVar[@]}" 


ecno 


# ^ + space tab newline 


echo '---Case6: Within double-quotes IFS set and empty ' 


TESZ A 

echo "Here is: printf $q "(S(ArrayVar[*])"' 
prasie aor Ve eese Wise [| 751] jp? 

echo 
Scho “kero 183 princi fep UJ Sd Xue wea lQ] 
printf %q "S${ArrayVar[@]}" 


echo 

cho 'Here is: echo "S{ArrayVar[*]}"' 
echo "S${ArrayVar[@]}" 

cho 'Here is: echo "{${ArrayVar[@]}"! 


echo "S${ArrayVar[@]}" 


echo 

echot Case e Mehin Clowloslla-cuores = Iles se wiser Y 
asec ILS 

echo 'Here is: printf $q "{${ArrayVar[*]}"' 

oeae ye; Uist Meret Velie [| o5 ]| d 

echo 
echo 'Here is: printf %q "{${ArrayVar[@]}"' 
printf Sq "${ArrayVar[@]}" 


echo 

cho 'Here is: echo "S{ArrayVar[*]}"' 
echo "S${ArrayVar[@]}" 

cho 'Here is: echo "{${ArrayVar[@]}"! 


echo "S${ArrayVar[@]}" 


echo 
Sele, ==! (oue Coes 


cimo Ww 


# Put IFS back to the default. 
# Default is exactly these thr bytes. 
TES =S uN ZON 5 U Nos (0) ) Y Ns, U # In exactly this 


# Interpretation of the above outputs: 


# A Globo Paces as 1/03 TAE SeSiciealine; QE MPS aneleiceies o 


HEH 


order. 


cho 


346 An All-Elements-Of does not consider IFS settings. 

347 ### 

348 Note the different output using the echo command and the 
349 #+ quoted format operator of the printf command. 

350) 

35i 

3152 Recall: 

353 Parameters are similar to arrays and have the similar behaviors. 
354 # 

355) The abov xamples demonstrate the possible variations. 
356 To retain the shape of a sparse array, additional script 
357 #+ programming is required. 

S58 # 

EDI The source code of Bash has a routine to output the 

360 #+ [subscript]=value array assignment format. 

Siow! As of version 2.05b, that routine is not used, 

362 #+ but that might change in future releases. 

363 

364 

365 


366 # The length of a string, measured in non-null elements (characters): 
367 echo 


368 echo ' Non-quoted references ! 

369 echo 'Non-Null character count: '${#VarSomething}' characters.' 
370 

SL mw» TSS EH! Wale SY YOO)? Veer il u^ SYNXOO! sus a il Claeicacieeue s 
372 4 eeho test! # See that? 

373 

374 

975 

376 The length of an array, measured in defined elements, 

377 #+ including null content elements. 

378 echo 


379 echo 'Defined content count: 'S${#ArrayVar[@]}' elements.' 
380 That is NOT the maximum subscript (4). 


381 That is NOT the range of the subscripts (1. . 4 inclusive). 
382 It IS the length of the linked list. 

383 ### 

384 Both the maximum subscript and the range of the subscripts may 
385 #+ be found with additional script programming. 

386 

25d The length of a string, measured in non-null elements (characters): 
388 echo 

389 echo “ Quoted, Glob-Pattern references : 

390 echo 'Non-Null character count: '"${#VarSomething}"' characters.' 
Sil 


392 # The length of an array, measured in defined elements, 
393 #+ including null-content elements. 


394 echo 

395 echo 'Defined element count: '"${#ArrayVar[*]}"' elements.' 

396 

Sog Gz WinheSicjoretsicehe soils SWesiemMewresoin Clos imo erioet Gha Sis sag |) (9j9XeuctenE To, 


398 # Suggestion: 
399 # Always use the All-Elements-Of character 


400 #+ if that is what is intended (independence from IFS). 
401 

402 

403 

404 Define a simple function. 

405 I include an underscore in the nam 

406 #+ to make it distinctive in th xamples below. 
407 # 

408 Bash separates variable names and function names 
409 #+ in different namespaces. 

410 The Mark-On yeball isn't that advanced. 

411 # 


[Es qi HES des dms qe qf qm Ex ghi dy es des SS qüms qs qíes SS qim qus qWes qi qu gimy qus qe des ies SS qi (iy (ies qe fes qhey SS qm qms qíus qe. ES ey ghe. quy es qieso pes Ses des qe SS Re SS SSS qim qiue qm qe qe qns 


—simple() { 
echo -n 'SimpleFunc'$G 


#  Newlines are swallowed in 
#+ result returned in any case. 


w duae ( goo J aexseunailo)o GimnvOkeS a (lownueuo(e| (oue PUNCE. 
# The $( ... ) notation is pronounced: Result-Of. 


# Invoke the function simple 
echo 


Scho Y= = OUOUIE (Qi PUMcELOM Simple = =! 


_simple 
echo 

Wr (ue 

( simple) 
echo 


Chow l= re theres a "wteucsieloley ou Taar 
echo $ simple not defined 


## 
$( simple) 
line 436: 


# Try passing arguments. 


# Try passing arguments. 


name? -' 
# No variable by that name. 


Invoke the result of function simple (Error msg intended) 


# Gives an error messag 
SimpleFunc: command not found 


echo 
## 


LE 

This demonstrates that the output 
## 

Interpretation: 


The first word of the result of function _simple 
+ is neither a valid Bash command nor the name of a defined function. 


of _simple is subject to evaluation. 


## 
—jeseshmwE () 4i 
exo -n Voici Sep "SB 


A function can be used to generate in-line Bash commands. 


A simple function where the first word of result IS a bash command: 


eco t= = OUFEOUIES: Oi evni ON orne = =! 


_print parml parm2 
echo 


$( print parml parm2) 


echo 


$( print $VarSomething) 
echo 


# Function variables 
# 


echo 
exeo. Y= = BUNCE doOM Wescrelolles = =! 
# A variable may represent a signed 


An Output NOT A Command. 


Executes: printf %q parml parm2 
S above IFS examples for the 
+ various possibilities. 


The predictable result. 


integer, a string or an array. 


Gab Gal Gn) Gay Gay ON OUS 


oul 
ND + 
Sy XO) 


# A string may be used like a function name with optional arguments. 


# set -vx 
declare -f funcVar 


funcVar=_print 
SfuncVar parml 
echo 


funcVar-$( print ) 
SfuncVar 

SfuncVar $VarSomething 
echo 


funcVar-$( print $VarSomething) 
$funcVar 
echo 


funcVar-"$( print $VarSomething)" 
SfuncVar 
echo 


Enable if desired 
+ in namespace of functions 


Contains name of function. 
Same as _print at this point. 


Contains result of function. 
No input, No output. 
The predictable result. 


$VarSomething replaced HERE. 
The expansion is part of the 
+ variable contents. 


$VarSomething replaced HERE. 
The expansion is part of the 
+ variable contents. 


# The difference between the unquoted and the double-quoted versio 
#+ above can be seen in the "protect literal.sh" example. 


ns 


# The first case above is processed as two, unquoted, Bash-Words. 
# The second case above is processed as one, quoted, Bash-Word. 
# Delayed replacement 
# 
echo 
Eine " Delayed replacement j 
funcVar-"$( print '$VarSomething')" 4 No replacement, single Bash-Word. 
eval $funcVar # SVarSomething replaced HERE. 
echo 
VarSomething-'NewThing' 
eval $funcVar # SVarSomething replaced HERE. 


echo 


# Restore the original setting trashed above. 


VarSomething-Literal 


# There are a pair of functions demonstrated in the 

#+ "protect literal.sh" and "unprotect literal.sh" examples. 

# These are general purpose functions for delayed replacement lit 
#+ containing variables. 


* (in concept, anyway). 
# 


+ Bash-Array: array_name. 
# 


A string can be considered a Classic-Array of elements (characte 
A string operation applies to all elements (characters) of the string 


[The notation: $(array name[80]) represents all elements of the 


he Extended-Syntax string operations can be applied to all 


Talg 


ESIE 


544 
545 
546 
547 
548 
549 
550 
5/51 
552 
393 
554 
555 
556 
95 
558 
559 
560 
561 
562 
3/53) 
564 
565 
566 
567 
568 
569 
570 
Su 
512 
S73 
574 
5718) 
576 
577 
578 
EX 
580 
581 
582 
583 
584 
SGS 
586 
SOT 
588 
589 
590 
S9 
592 
9293 
594 
295 
596 
597 
598 
997 
600 
601 
602 
603 
604 
605 
606 
607 
608 
609 


* elements of an array. 
# 
This may be thought of as a For-Each operation on a vector of strings. 
# 
Parameters are similar to an array. 
The initialization of a parameter array for a script 
+ and a parameter array for a function only differ 
+ in the initialization of ${0}, which never changes its setting. 
# 
Subscript zero of the script's parameter array contains 
+ the name of the script. 
# 
Subscript zero of a function's parameter array DOES NOT contain 
+ the name of the function. 
The name of the current function is accessed by the SFUNCNAME variable. 
A quick, review list follows (quick, not short). 
echo 
SEIN Y= = USSic Duti noti echange) c = 
echo '- null reference -' 
echo -n ${VarNull-'NotSet'}' ' NotSet 
echo ${VarNull} NewLine only 
echo n $(VarNull:-'NotSet')' ' NotSet 
echo ${VarNull} Newline only 
@elne Y= mL Cweimeeimes. =! 
Scho =i SbweuctwexEs- mpg" “ Only the space 
echo ${VarEmpty} Newline only 
Ean =n S (Vadnmo Vinay bY V Empty 
echo ${VarEmpty} Newline only 
Giao. '—= Comcames =! 
echo $(VarSomething-'Content') Literal 
echo $(VarSomething:-'Content'j Literal 
Giao v= Hoarse Aiewehy =" 
echo ${ArrayVar[@]-—'not set'} 
ASCII-Art time 
Sta W==yes,;, eme 
Unset Y Y Siu ss. | = 0 
Empty N 3 S{# ... } == 0 
Contents N N Siu we dw O 
Either the first and/or the second part of the tests 
+ may be a command or a function invocation string. 
echo 
GEN Yo = USS I Corn Uincleitnmecl — =! 
declare -it 
CSE) 
t=$t-1 
} 
# Null reference, set: t == -1 
t=S{#VarNull1} s Ie sli - ZrO. 
${VarNull- _decT } # Function executes, t now -1. 
echo St 
42 INULLIL contents, sete ic == © 
t=${#VarEmpty} sr Results im ZSrG» 
$(VarEmpty- | decT } # _decT function NOT executed. 
Scho Sr 
# Contents, set: t == number of non-null characters 


Oar OSO Or Or Or O O 
(69) S| @y (rl Wem 163) MS) [Se 


OO 
N H H 
(y AG) 


Set to valid function name. 
non-zero length 

Function simple executed. 
Note the Append-To action. 


VarSomething-' simple' 
t=${#VarSomething} 
$(VarSomething- _decT } 
echo $t 


Se oc cho dk 


# Exercise: clean up that example. 
LIASSIE T 

unset | decT 

VarSomething-Literal 


echo 
eleg). Y= = WSs duse! Change = = 

echo '- Assignment if null reference -' 

echo m S varne iS NOEScG M # NotSet NotSet 
echo ${VarNull} 


unset VarNull 


echo '- Assignment if null reference -' 

exeo. m (S weisenmedlils=VinoneSere VY v 4 NotSet NotSet 
echo ${VarNull} 

unset VarNull 


echo '- No assignment if null contents -' 

echo -n ${VarEmpty='Empty'}' ' # Space only 
echo ${VarEmpty} 

VarEmpty-'' 

echo '- Assignment if null contents -' 

echo -n S${VarEmpty:='Empty'}' ' # Empty Empty 
echo ${VarEmpty} 

VarEmpty-'' 

echo '- No change if already has contents -' 

echo ${VarSomething='Content'} # Literal 
echo ${VarSomething:='Content' } # Literal 


"Subscript sparse" Bash-Arrays 


Bash-Arrays are subscript packed, beginning with 
+ subscript zero unless otherwise specified. 


The initialization of ArrayVar was one way 


+ to "otherwise specify". Here is the other way: 
echo 

declare -a ArraySparse 
ArraySparse-( [1]-on ele" [2S tious 


# [0]2null reference, [2]=null content, [3]=null referenc 


Chio " Array-Sparse List ! 
# Within double-quotes, default IFS, Glob-Pattern 


IFS=$'\x20'S'\x09'S'\x0A' 
printf Sq "${ArraySparse[*]}" 
echo 


Note that the output does not distinguish between "null content" 
+ and "null reference". 
Both print as escaped whitespac 
# 
Note also that the output does NOT contain escaped whitespac 
+ for the "null reference(s)" prior to the first defined element. 
# 
This behavior of 2.04, 2.05a and 2.05b has been reported 
+ and may change in a future version of Bash. 


676 
671 
678 
GTS 
680 
681 
682 
683 
684 
685 
686 
687 
688 
689 
690 
59 
692 
693 
694 
695 
696 
697 
698 
$99 
700 
701 
702 


# To output a sparse array and maintain the [subscript]=value 
#+ relationship without change requires a bit of programming. 
# One possible code fragment: 


ttt 


# local 1=${#ArraySparse[@] } 
# local f=0 
# local i=0 


( 


# 
N 


# 


for 
do 


done 


OTSE 


Exercise: 


(( 1=${#ArraySparse[@]}, 


# ‘if defined 


S${ArraySparse[$i]+ eval echo 


The reader coming upon th 
+ might want to review "command lists" and "multiple commands on a line" 
+ in the text of the foregoing "Advanced Bash Scripting Guide." 


unset ArraySparse 


ecno 
ecno 
ecno 
ecno 
ecno 


ecno 


| x = Conch enone alleeieaeice 


then... 


abov 


# Count of defined elements 
# Count of found subscripts 
# Subscript to test 

# Anonymous in-line function 


0, 


"X 


qe EE MET EY 


['Si']='${ArraySparse [$i] } 


clean it up. 


code fragment cold 


The "read -a array name" version of the "read" command 
+ begins filling array name at subscript zero. 
ArraySparse does not define a value at subscript zero. 


The user needing to read/write a sparse array to either 
+ external storage or a communications socket must invent 
+ a read/write code pair suitable for their purpose. 


Gawie moe Change) 
w= ING alrerieare ii inullil werenmenee =! 

-n ${VarNull+'NotSet'}' 
${VarNull} 
unset VarNull 


i= Ne esee. alice goil ieeureiceince. =! 


echo -n ${VarNull:+'NotSet'}' 
echo ${VarNull} 
unset VarNull 


cno 


Alternat 


echo -n ${VarEmpty+'Empty'}' 
echo ${VarEmpty} 


VarEmpty-'' 


ecno 


sir MELL Gemireimcs =! 


No alternats ii mll Content 


echo -n ${VarEmpty:+' 
echo ${VarEmpty} 
VarEmpty-'' 


cno 


Alternat 


# Alternate literal 
echo -n ${VarSomething+'Content'}' ' 
echo ${VarSomething} 


# Invoke function 
echo -n ${VarSomething:+ $(_simple) 
echo ${VarSomething} 


ecno 


Empty'J' 


) ' 


# Empty 


# Space only 


if already has contents -' 


# Content Literal 


# SimpleFunc Literal 


, 


(( 


ftt 


)) 


} 


742 echo ' Sjocussie Aieweny — = 


743 echo ${ArrayVar[@]+'Empty'} # An array of 'Empty' (ies) 
744 echo 

745 

TAG exo Y= — eSt 2 tor wackitimecl = =! 

747 


748 declare -i t 

JAS sinew) 4 

750 t-S$t41 

ASL A 

TOA 

753 # Note: 

754 # This is the same test used in the sparse array 

755 #+ listing code fragment. 

756 

Ja? x NOI westenence, Herh ic == il 

758 t=${#VarNull}-1 # Results in minus-one. 
759 S{VarNull+ .incT } # Does not execut 

760 echo $t' Null reference' 

761 
762 # Null contents, set: t == 0 

763 t=${#VarEmpty}-1 # Results in minus-one. 
764 S{VarEmpty+ _incT } # Executes. 

765 echo $t' Null content' 

766 
767 4 Contents, set: t == (number of non-null characters) 

768 t=${#VarSomething}-1 # non-null length minus-one 
769 S{VarSomething+ | incT } # Executes. 

770 echo $t' Contents' 

UAL 
772 # Exercise: clean up that example. 
773 unset t 

TVA cwuoyekewE, imer 


3358) 

776 # S(name?err msg) $(name:?err msg) 

777 # These follow the same rules but always exit afterwards 
778 #+ if an action is specified following the question mark. 
779 # The action following the question mark may be a literal 
VEO Gar Or 2 wauUincicmem ifi s 

781 ### 


782 # S{name?} S{name:?} are test-only, the return can be tested. 
783 

784 

785 

786 

787 # Element operations 

788 # 
789 
790 echo 

791 echo '- - Trailing sub-element selection ! 
792 

793 # Strings, Arrays and Positional parameters 
794 

795 # Call this script with multiple arguments 
TIG ir tO B the parameter selections. 

VOY 

TIS «exe. Ve Wakil =¥ 


799 echo ${VarSomething:0} # all non-null characters 

800 echo ${ArrayVar[@]:0} # all elements with content 

801 echo $(8:0) # all parameters with content; 
802 # ignoring parameter[0] 

803 

804 echo 

(X045) clave; Ve JAMLIL queue sU 

806 echo ${VarSomething:1} # all non-null after character[0] 


807 echo ${ArrayVar[@]:1} # all after element[0] with content 


(00) @o} (eey Yo) 2). (oor OO SO 
(9) sal ox yl des Go [S95 


œ OO 
NO + 
COMES 


echo ${@:2} # all after param[1] with content 


echo 
clo “= Neuere cutie =" 
echo ${VarSomething:4:3} * ral 
# Three characters after 
5s (meus [L3] 
Cho “= Sparse array Goren —! 
echo ${ArrayVar[@]:1:2} # four The only element with content. 
# Two elements after (if that many exist). 
# the FIRST WITH CONTENTS 
#+ (the FIRST WITH CONTENTS is being 
#+ considered as if it 
#+ were subscript zero). 
# Executed as if Bash considers ONLY array elements with CONTENT 


# printf $q "${ArrayVar[@]:0:3}" # Try this one 


ji; dial Verano 250. A OSei «uae, 2 OSIe, 
#+ Bash does not handle sparse arrays as expected using this notation. 


# The current Bash maintainer, Chet Ramey, has corrected this. 


Ceo. '—= Nor -spatse sume. =" 
echo S103262) # Two parameters following parameter [1] 
# New victims for string vector examples: 


stringZ=abcABC123ABCabc 
arrayZ-( abcabc ABCABC 123123 ABCABC abcabc ) 


ejos IHleVsleeesc" 3l]evswexemweo" JA] qpu»jev3b25123" ) 
echo 
Secho V = = Wal@ibalin iene, = SXsyETcaLiNSj A = | 
eeno " = = yicrtin array = -V Siarcrayal@li = = v 
Cino v Sparse array = = Si sjoarsen je] = = v 
che. y [0] null ref, [2]==null ref, [4]==null content - ' 
echo ' - [1]=abcabc [3]=ABCABC [5]=123123 - ' 
Chom non-null-reference count: 'S${#sparseZ[@]}' elements' 
echo 
echo Ve. = Brari SUo-e lerene removal ! 
(soy Glob-Pattern match must include the first character. — = 
Cno y Glob-Pattern may be a literal or a function result. — —' 
echo 
# Function returning a simple, Literal, Glob-Pattern 


alse ( d 
echo n aber 

} 

cho fSimoritextE Jastix =! 
echo ${stringZ#123} # Unchanged (not a prefix). 
echo $(stringZ4S$( abc))] # ABCI23ABCabc 
echo S{arrayZ[@]#abc} # Applied to each element. 
# echo S${sparseZ[@]#abc} # Version-2.05b core dumps. 


# Has since been fixed by Chet Ramey. 


# The -it would be nic Emoe Suisses - (Qt 


# echo S${#sparseZ[@] #*} # This is NOT valid Bash. 
echo 

Glao. Y= Towers [oet =" 
echo ${stringZ##1*3} # Unchanged (not a prefix) 


MO XO XO XO XO XO O 
(99) s io» rl dex Go WS) [3 


MO xo 
NO +t H 
SRO} 


echo S${stringZ##a*C} # abc 
echo S${arrayZ[@]##a*c} sr ABCADC 125123 ABCABC 
# echo ${sparseZ[@]##a*c} # Version-2.05b core dumps. 
# Has since been fixed by Chet Ramey. 
echo 
echo '- - Suffix sub-element removal ! 
laxo V Glob-Pattern match must include the last character. - -' 
gag # Glob-Pattern may be a literal or a function result. - -' 
echo 
ee). Y= Sys E Sisters =! 
echo S$S[stringZ$1*3) # Unchanged (not a suffix). 
echo S${stringZ%$ (_abc) } # abcABC123ABC 
echo S{arrayZ[@]%abc} # Applied to each element. 
# echo S${sparseZ[@]%abc} # Version-2.05b core dumps. 
# Has since been fixed by Chet Ramey. 
# The -it would be nice- Last-Subscript-Of 


# echo S${#sparseZ[@] %*} 


ec 
ec 
ec 
ec 
ec 


# echo ${sparseZ[@]%%b*c} 


NO 
NO 
NO 
NO 
NO 


# This is NOT valid Bash. 


Y= WINS SSE SUE PIE 


$(stringz$$1*3) 


# Unchanged (not a suffix) 


$(stringZz$$b*c) # a 
S{arrayZ[@]%%b*c} # a ABCABC 123123 ABCABC a 


# Version-2.05b core dumps. 


lement at any location in string. - -' 


Glob-Pattern may be a literal or Glob-Pattern function result. 


nd specification may be a literal or function result. 
nd specification may be unspecified. Pronounce that' 


# Has since been fixed by Chet Ramey. 
echo 
gio ¥ Sub-element replacement Y 
cho ' Sub 
eeo Y= = mirst Syeexciitieuenigum ws a (ulolo-ietiesm — =! 
Chom 
Sela; Y= — fex 
echo '- - Seco 
echon as: Replace-With-Nothing (Delete) - -' 
echo 


# Function returning a simple, Literal, Glob-Pattern 


1230) 


{ 


Seino =i VE 


29 


no 
no 
no 
no 
NO 


NO 


Elorrio S 123) 7 S99 Y 
${stringZ/ABC/xyz} 
S{arrayZ[@]/ABC/xyz} 
S{sparseZ[@]/ABC/xyz} 


Replac 


# 


NO 
NO 
no 
no 
no 


Th 


Delet 


ELERE (xeloiuczemee =! 

Changed (123 is a component) . 
xyZABC123ABCabc 

Applied to each element. 
Works as expected. 


Se SE OH OSE 


PICS OCEellicicsnces =! 


S Lesen S C 3:2 3» /} 
${stringZ/ABC/} 
${arrayZ[@]/ABC/} 
S{sparseZ[@]/ABC/} 


replac 


ment need not be a literal, 


#+ since the result of a function invocation is allowed. 


# 


echo 


[This is general to all forms of replacement. 


cho 


Replac 


ELESIE OSCGUIEMSMGS yie REésullic-—Oi =" 


940 
941 
942 
943 
944 
945 
946 
947 
948 
949 
950 
951 
952 
953) 
954 
955 
956 
95) 
958 
959 
960 
961 
962 
963 
964 
965 
966 
BIS 
968 
969 
970 
9 3t 
Op 
9 18) 
974 
975 
976 
Qe 
978 
97$ 
980 
981 
982 
983 
984 
MIS 
986 
987 
988 
989 
990 
9t 
992 
993 
994 
S 
996 
9297 
998 


NO 
NO 
no 


NO 


${stringZ/$(_123)/$(_simple)} # 
S{arrayZ[@]/ca/$(_simple) } # 
S{sparseZ[@]/ca/S (_simple) } # 


NO 
NO 
NO 
NO 
no 


no 


"— Replace all occurrences -' 
$(stringZ//[b2]/X]) 
$(stringZ//abc/xyz) 
S{arrayZ[@]//abc/xyz} 
S{sparseZ[@]//abc/xyz} 


Se SF ch SHE 


NO 
no 
no 
no 
no 


no 
no 
no 
no 


'— Delete all occurrences -' 
$(stringZ//[b2]/] 
$(stringZ//abc/) 
S{arrayZ[@]//abc/} 
${sparseZ[@]//abc/} 


Wo 
Ap 
Wo 


Xx- 
xy 
Ap 
Wo 


rks as expected. 
plied to each element. 
rks as expected. 


owe los mul 2's 
ZABC123ABCxyz 

plied to each element. 
rks as expected. 


'- — Prefix sub-element replacem 


fit 


i= — Anselm, mese meludie tis icis eur. Cl~isaceee, = =! 


no 
NO 
NO 
NO 
no 


no 


'- Replace prefix occurrences -' 
$(stringZ/4[b2]/X]) 
${stringZ/#$ (_abc) /XYZ} 
S{arrayZ[@]/#abc/XYZ} 
S{sparseZ[@]/#abc/xXYZ} 


Se HE OSE OSE 


NO 
no 
no 
no 
no 


NO 
NO 
NO 
no 


'- Delete prefix occurrences -' 
${stringZ/#[b2]/} 

S{stringZ/#$ (_abc) /} 
S{arrayZ[@]/#abc/} 
${sparseZ[@]/#abc/} 


Un 
EXT 
Ap 
Wo 


changed (neither is a prefix). 
ZABC123ABCabc 

plied to each element. 
rks as expected. 


'- — Suffix sub-element replacem 
i= — Match must include the last 


no 
no 
no 
NO 
no 


no 


'- Replace suffix occurrences -' 
$(stringZ/$[b2]/X]) 
${stringZ/%$ (_abc) /XYZ} 
S{arrayZ[@]/%abc/XYZ} 
S{sparseZ[@]/%abc/XYZ} 


Se SE ch dk 


Oe 


NO 
NO 
NO 
no 
NO 


NO 
NO 
NO 


NO 


I= DEES SUE CeCuieceaices 1 
Si eese sta 5 992 1 / } 

$(stringZ/$$( abc)/) 
S{arrayZ[@]/%abc/} 
${sparseZ[@]/%abc/} 


ME 
(cin. 


Un 
ab 
Ap 
Wo 


APACE CIE M E 


changed (neither is a suffix). 
CABC123ABCXYZ 

plied to each element. 
rks as expected. 


t= = SOSCieil Cases o ull Cllolo-wWaicicenmn = =! 


Vi jeseeueise eli =! 


999 # null substring pattern means 'prefi 
echo ${stringZ/#/NEW} 

echo ${arrayZ[@]/#/NEW} 

echo ${sparseZ[@]/#/NEW} 


1000 
1001 
1002 
1003 
1004 
1005 


echo 


EWabcABC123ABCabc 
pplied to each element. 


hat seems reasonable. 


1006 echo Y= Suis alll =" 

007 # null substring pattern means 'suffix' 

008 echo ${stringZ/%/NEW} # abcABC123ABCabcNEW 

009 echo ${arrayZ[@]/%/NEW} # Applied to each element. 
010 echo ${sparseZ[@]/%/NEW} # Applied to null-content also. 
QLI # That seems reasonable. 
(3.2 

013 echo 

014 echo '- - Special case For-Each Glob-Pattern - -' 

015 echo V— = = = Imis is a MUGS -rO-Imewea chee ! 

016 echo 

91.7 

018 _GenFunc() { 

019 echo -n ${0} # Illustration only. 

020 # Actually, that would be an arbitrary computation. 

021 } 

(12:22 


023 # All occurrences, matching the AnyThing pattern. 

024 # Currently //*/ does not match null-content nor null-reference. 
025 4 /#/ and /$/ does match null-content but not null-reference. 
026 echo $(sparseZz[G]//*/$( GenFunc)) 


lA p RB p B pH p HB p p|B p pH pH pP H pH PB pP PH H pPH H H B HP H H pm|B REP EE 


027 

028 

029 A possible syntax would be to make 

030 #+ the parameter notation used within this construct mean: 
031 $11) The full element 

(132 $(2) - The prefix, if any, to the matched sub-element 

033 ${3} The matched sub-element 

034 $(4) - The suffix, if any, to the matched sub-element 

035 

036 echo $(sparseZ[G]//*/S$( GenFunc ${3}) } # Same as $(1) here. 
037 Perhaps it will be implemented in a future version of Bash. 
038 

039 


1040 exit O0 
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Appendix B. Reference Cards 


The following reference cards provide a useful summary of certain scripting concepts. The foregoing text 
treats these matters in more depth, as well as giving usage examples. 


Table B-1. Special Shell Variables 


$0 Filename of script 


$ Last argument of previous command 
$! Process ID (PID) of last job run in background 


* Must be quoted, otherwise it defaults to "$@". 


Table B-2. TEST Operators: Binary Comparison 


Operator Meaning [Operator [Meaning 


Fea — Bqualto | Bua to 
Egat 

Ene —— (Notequalto t= Notequalto 
put qpesthan | [Lessthan(ASCID * 
Ele — essthanorequalto | | 

Hot Greaterthan | Greater than (ASCID * 


-Z String is empty 

-n String is not empty 
Arithmetic Comparison |within double parentheses (( ... )) |] | 
Bo Greaterthan 0 S 


>= Greater than or equal to poc 
E kesma o ooo |o 


<= Less than or equal to E es 


* [f within a double-bracket [[ ... || test construct, then no escape \ is needed. 


Table B-3. TEST Operators: Files 


Operator {Tests Whether .—- Operator Tests Whether 
ENT 


File exists 


File exists O 

Er ^ Fileisaregularfie | J | O 
-h File is a symbolic link -wW File has write permission 

EL [File is a symboliclink | f» [File has execute permission | 

a ee 


-b fil is a block device a 
-c fie is a character device | | -g sgid flag set 


File is not zero size 


Hp Fileisapipe | þf suid flag set 
S File is a socket | [l-k "sticky bit" set 
-t [File is associated with averminal| | | 


N File modified since it was last Bi -nt F2 file Fl is newer than F2 * 
read 


You own the file | Fi -ot F2 File Fl is older than F2 * 


-G Group id of file same as yours F1 -ef F2|Files Fl and F2 are hard links to the same 
file * 
l 


NOT (inverts sense of above 
tests) 


* Binary operator (requires two operands). 


Table B-4. Parameter Substitution and Expansion 


Expression Meaning 
Stvar} [Value of var, same as $var 


$ 
${var-DEFAULT} |If var not set, evaluate expression as SDEFAULT * 
$ : 


var:-DEFAULT) If var not set or is empty, evaluate expression as SDEFAULT * 


S(var-DEFAULT) |If var not set, evaluate expression as SDEFAULT * 
$(var:-DEFAULT) |If var not set, evaluate expression as SDEFAULT * 


S{var+OTHER} If var set, evaluate expression as SOTHER, otherwise as null string 


var?El 


vari 2] 


S{var:+OTHER} If var set, evaluate expression as SOTHER, otherwise as null string 


Is{ !varprefix*) [Matches all previously declared variables beginning with varprefix 
${!varprefix@} |Matches all previously declared variables beginning with varprefix 


* Of course if var is set, evaluate the expression as Svar. 


Table B-5. String Operations 


Meaning 


Biéstring) Length of $string 


tring#substring} 


$ 
$ 


S 


trin 


string/substring/replacement } 


g//substring/replacement 


S tst 
S{string:position} Extract substring from $stringat Sposition 


tring:position: length} 


Extract $length characters substring from $string 
at Sposition 


Strip shortest match of S$subst ring from front of 
$string 


$string 
$string 
S! g 


$$substring) 


Strip longest match of $substring from back of 
$string 


Replace first match of Ssubst ring with 
Sreplacement 
Replace all matches of Ssubst ring with 
Sreplacement 


g/#substring/replacement 


g/ssubstring/replacement 


If Ssubst ring matches front end of $string, 


expr match "Sstring" '$substring' Length of matching $substring* at beginning of 
$string 


expr "Sstring" 


expr index "Sst 


expr substr Sst 
Slength 


: 'Ssubstring' 


tring" Ssubstring 


tring $position 


Sstring 

Numerical position in $st ring of first character in 
Ssubstring that matches 

at Sposition 


expr match "$string" Extract Ssubst ring* at beginning of $string 
'\ (Ssubstring\) ' 


expr "$string" : '\(Ssubstring)\) ' Extract Ssubst ring* at beginning of $string 


expr match "Sstring" Extract Ssubstring* atend of $string 
'.*\ (Ssubstring\)' 


expr "$string" : '.*\(Ssubstring\) ' |Extract Ssubstring* atend of $string 


* Where Ssubst ring is a Regular Expression. 


Table B-6. Miscellaneous Constructs 


Brackets 

if [ CONDITION ] 
if [[ CONDITION ]] 
Array[1]=elementl 


[a-z] 


commandl; command2; . . . commandN; 


stringl,string2,string3,...] 


commandi1; command2 
Array-(element]l lement2 element3) 
result-$ (COMMAND) 
» (COMMAND) 
« (COMMAND) 


var = 78 )) 
var-$(( 20 + 5 )) 
(( vart* )) 
(( wat-- jJ 
varO = var1«98?9:21 )) 


"Weak" quoting 


Ustring' Strong" quoting 


Back Quotes Is >= = = 
result-' COMMAND ~ Command substitution, classic style 
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Appendix C. A Sed and Awk Micro-Primer 


This is a very brief introduction to the sed and awk text processing utilities. We will deal with only a few 
basic commands here, but that will suffice for understanding simple sed and awk constructs within shell 
scripts. 


sed: a non-interactive text file editor 

awk: a field-oriented pattern processing language with a C-style syntax 

For all their differences, the two utilities share a similar invocation syntax, use regular expressions , read input 
by default from st din, and output to stdout. These are well-behaved UNIX tools, and they work together 
well. The output from one can be piped to the other, and their combined capabilities give shell scripts some of 


the power of Perl. 


<p>) One important difference between the utilities is that while shell scripts can easily pass arguments 
to sed, it is more complicated for awk (see Example 35-5 and Example 28-2). 


C.1. Sed 


Sed is a non-interactive [1] stream editor. It receives text input, whether from st din or from a file, performs 
certain operations on specified lines of the input, one line at a time, then outputs the result to stdout or toa 
file. Within a shell script, sed is usually one of several tool components in a pipe. 


Sed determines which lines of its input that it will operate on from the address range passed to it. [2] Specify 
this address range either by line number or by a pattern to match. For example, 3d signals sed to delete line 3 
of the input, and /Windows/d tells sed that you want every line of the input containing a match to 
"Windows" deleted. 


Of all the operations in the sed toolkit, we will focus primarily on the three most commonly used ones. These 
are printing (to stdout), deletion, and substitution. 


Table C-1. Basic sed operators 


Operator Name Effect 
[address-range]/p print Print [specified address range] 


[address-range]/d delete — [Delete [specified address range] 


s/patternl/pattern2/ substitute |Substitute pattern2 for first instance of 
pattern! in a line 
[address-range]/s/patternl/pattern2/ substitute Substitute pattern2 for first instance of 
pattern! in a line, over address-range 


[address-range]/y/patternl/pattern2/ transform |replace any character in patternl with the 
corresponding character in pattern2, over 
address-range (equivalent of tr) 


global Operate on every pattern match within each 
matched line of input 


ə Unless the g (global) operator is appended to a substitute command, the substitution operates only on the 
first instance of a pattern match within each line. 
From the command-line and in a shell script, a sed operation may require quoting and certain options. 


1 sed -e '/*S/d' $filename 

2 # The -e option causes the next string to be interpreted as an editing instruction. 
3 # (If passing only a single instruction to sed, the "-e" is optional.) 

4 # The "strong" quotes ('') protect the RE characters in the instruction 

5 #+ from reinterpretation as special characters by the body of the script. 

6 # (This reserves RE expansion of the instruction for sed.) 

7 # 


8 # Operates on the text contained in file $filename. 


In certain cases, a sed editing command will not work with single quotes. 


1 filename=filel.txt 

2 pattern=BEGIN 

3 

4 sed "/*Spattern/d" "Sfilename" # Works as specified. 

5 4 sed '/*Spattern/d' "$filename" has unexpected results. 
6 # lin MLS auwepeua c, Whitin strone epuotime (" ss. Vy 

7 


#+ "Spattern" will not expand to "BEGIN". 


Sed uses the —e option to specify that the following string is an instruction or set of instructions. If 
there is only a single instruction contained in the string, then this may be omitted. 


1 sed -n '/xzy/p' $filename 

2 4 The -n option tells sed to print only those lines matching the pattern. 

3 4 Otherwise all input lines would print. 

4 # The -e option not necessary here since there is only a single editing instruction. 


Table C-2. Examples of sed operators 


Substituting a zero-length string for another is equivalent to deleting that string within a line of input. This 
leaves the remainder of the line intact. Applying s/GUI// to the line 


The most important parts of any application are its GUI and sound effects 

results in 

The most important parts of any application are its and sound effects 

A backslash forces the sed replacement command to continue on to the next line. This has the effect of using 
the newline at the end of the first line as the replacement string. 


1 g^ S/A 

2 IS 
This substitution replaces line-beginning spaces with a newline. The net result is to replace paragraph indents 
with a blank line between paragraphs. 


An address range followed by one or more operations may require open and closed curly brackets, with 
appropriate newlines. 


1 /[0-9A-Za-z]/,/^$/( 
2 M R/E 
S 


This deletes only the first of each set of consecutive blank lines. That might be useful for single-spacing a text 
file, but retaining the blank line(s) between paragraphs. 
£^; The usual delimiter that sed uses is /. However, sed allows other delimiters, such as %. This is useful 
when / is part of a replacement string, as in a file pathname. See Example 11-9 and Example 16-32. 


į ) A quick way to double-space a text file is sed G filename. 


For illustrative examples of sed within shell scripts, see: 


. Example 35-1 

. Example 35-2 

. Example 16-3 

. Example A-2 

. Example 16-17 
. Example 16-27 
. Example A-12 
. Example A-16 
. Example A-17 
. Example 16-32 
. Example 11-9 

. Example 16-48 
. Example A-1 

. Example 16-14 
. Example 16-12 
. Example A-10 
. Example 19-12 
. Example 16-19 
. Example A-29 
. Example A-31 
. Example A-24 
. Example A-43 


= = e i i i i 
OMAANNDNMNAPWNrFYK TOAMANAANHPWN 


NNN 
Neo 


For a more extensive treatment of sed, check the appropriate references in the Bibliography. 


Notes 


[1] Sed executes without user intervention. 
[2] If no address range is specified, the default is all lines. 
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C.2. Awk 


Awk [1] is a full-featured text processing language with a syntax reminiscent of C. While it possesses an 
extensive set of operators and capabilities, we will cover only a few of these here - the ones most useful in 
shell scripts. 


Awk breaks each line of input passed to it into fields. By default, a field is a string of consecutive characters 
delimited by whitespace, though there are options for changing this. Awk parses and operates on each separate 
field. This makes it ideal for handling structured text files -- especially tables -- data organized into consistent 
chunks, such as rows and columns. 


Strong quoting and curly brackets enclose blocks of awk code within a shell script. 


Sil is misle Fil, S2 ws misli pä, Cite, 


echo one two | awk '(print $1}!' 
one 


echo one two | awk '(print $2}! 
two 


But what is field 40 ($0)? 
echo one two | awk '(print $0J' 
one two 

All the fields! 


awk '(print $3)' Sfilename 
Prints field 43 of file $filename to stdout. 


ewe "Texesuo Sil SS Sp" Siew lencme 
Prints fields #1, #5, and #6 of file $filename. 


NPRPRPRPRP PPP PY 
O io 00 -1 O Oi i& (0. M. I. O xo 0 -1 O) O1 iS CQ) IO 5 


21 awk '(print $0)' $filename 
22 Diciimes theren irer iel 
23 Same effect as: Gat Sur:leuweWNS s 2 o OH a a o SEC "V Srileneme 


We have just seen the awk print command in action. The only other feature of awk we need to deal with here 
is variables. Awk handles variables similarly to shell scripts, though a bit more flexibly. 


1 ( total += $(column number) ) 
This adds the value of co1umn number to the running total of tota». Finally, to print "total", there is an 
END command block, executed after the script has processed all its input. 

JL END | prime total } 
Corresponding to the END, there is a BEGIN, for a code block to be performed before awk starts processing 
its input. 


The following example illustrates how awk can add text-parsing tools to a shell script. 


Example C-1. Counting Letter Occurrences 


! /bin/sh 
letter-count2.sh: Counting letter occurrences in a text file. 


Seri by myall |lmyed@woilla. tr] s 
Used in ABS Guide with permission. 
Recommented and reformatted by ABS Guide author. 


(exy (nl dEx (99) [s [3 


# 
# 
# 
# 
# 
# 


# Version 1.1: Modified to work with gawk 3.1.3. 
# (Will still work with earlier versions.) 


7 
8 
9 
0 
1 INIT TAB AWK-"" 

12 # Parameter to initialize awk script. 
13 count_case=0 
4 
5 
6 
7 
8 


FILE PARSE-$1 


E PARAMERR-85 


18 usage() 

19 4 

20 echo "Usage: letter-count.sh file letters" 2>é1 

2l # For example: ./letter-count2.sh filename.txt a b c 
22 exit SE PARAMERR # Too few arguments passed to script. 
23 i 

24 

2i ase || Jk sme SESAME Ye qgeXeno 

26 echo "$1: No such file." Axei 

29 usage # Print usage message and exit. 
2g. EL 

29 

SO ase [ zm USZV ]| p we 

E echo "$2: No letters specified." 2>61 

62 usage 

Sis} teal 

34 

35 eimi 4 Letters specified. 

36 for letter in "echo $@ # For each one 

37 do 

38 INIT_TAB_AWK="$INIT_TAB_AWK tab_search[${count_case}] = \ 
39 \USieceee\ Wo Pinal caol Steoume_ecase}!] = Up V 

40 # Pass as parameter to awk script below. 

41 count_case= expr S$count case + 1° 

42 done 

43 

44 DEBUG: 

45 echo SINIT TAB AWK; 

46 


47 cat SFILE PARSE | 


48 Pipe the target file to the following awk script. 
49 

50 

5l Earlier version of script: 


52 awk -v tab_search=0 -v final_tab=0 -v tab=0 -v \ 
59 nb letter-0 -v chara=0 -v chara2=0 \ 


55 awk N 

56 "BEGIN ( SINIT TAB AWK ) \ 

57 4 solace (\SO, tala, WA )p Y 

Bis} ror (ohara slink Tab) X 

59 4 sore (levara? imn TAD Searah A 
60 4 slik feao searcileharaAl == tableharal) T tinal caoleneraZl on P Pod OW 
SiL WIND { ror (ehara im tinal tab) \ 

G2 4t prine tao searchleharal] "WU => W final tablehnaral jk TU 


64 Nothing all that complicated, just 
65 #+ for-loops, if-tests, and a couple of specialized functions. 


Oy ed SP 
68 
69 4 Compare this script to letter-count.sh. 


For simpler examples of awk within shell scripts, see: 


. Example 15-14 
. Example 20-8 
. Example 16-32 
. Example 35-5 
. Example 28-2 
. Example 15-20 
. Example 29-3 
. Example 29-4 
. Example 11-3 
. Example 16-60 
. Example 9-16 
. Example 16-4 
. Example 10-6 
. Example 35-17 
. Example 11-8 
. Example 35-4 
. Example 16-53 


— 
CO ooc0o-1o!gtUt. Urt. -— 


- 
-O 


=. = e. me e 
- ON tA BW LD 


That's all the awk we'll cover here, folks, but there's lots more to learn. See the appropriate references in the 
Bibliography. 


Notes 


[1] Its name derives from the initials of its authors, Aho, Weinberg, and Kernighan. 
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Appendix D. Exit Codes With Special Meanings 


Table D-1. Reserved Exit Codes 


Example Comments 


Catchall for general errors let "varl = 1/0" Miscellaneous errors, such as "divide by 
zero" and other impermissible 
operations 


Misuse of shell builtins i Seldom seen, usually defaults to exit 
(according to Bash 
documentation) 


u 
126 Command invoked cannot Permission problem or command is not 
execute an executable 


code 1 


127 command not found" ^ [illegal command Possible problem with $PATH or a typo 

128 exit takes only integer args in the range 
0 - 255 (see first footnote) 

script 

ee Control-C is fatal error signal 2, (130 = 
128 + 2, see above) 

exit takes only integer args in the range 
0 - 255 


According to the above table, exit codes 1 - 2, 126 - 165, and 255 [1] have special meanings, and should 
therefore be avoided for user-specified exit parameters. Ending a script with exit 127 would certainly cause 
confusion when troubleshooting (is the error code a "command not found" or a user-defined one?). However, 
many scripts use an exit / as a general bailout-upon-error. Since exit code 1 signifies so many possible errors, 
it is not particularly useful in debugging. 


There has been an attempt to systematize exit status numbers (see /usr/include/sysexits.h), but this 
is intended for C and C++ programmers. A similar standard for scripting might be appropriate. The author of 
this document proposes restricting user-defined exit codes to the range 64 - 113 (in addition to 0, for success), 
to conform with the C/C++ standard. This would allot 50 valid codes, and make troubleshooting scripts more 
straightforward. [2] All user-defined exit codes in the accompanying examples to this document conform to 
this standard, except where overriding circumstances exist, as in Example 9-2. 


ə Issuing a $? from the command-line after a shell script exits gives results consistent with the table above 
only from the Bash or sh prompt. Running the C-shell or tcsh may give different values in some cases. 


Notes 


[1] Out of range exit values can result in unexpected exit codes. An exit value greater than 255 returns an 
exit code modulo 256. For example, exit 3509 gives an exit code of 225 (3809 % 256 = 225). 


[2] An update of /usr/include/sysexits.h allocates previously unused exit codes from 64 - 78. It 
may be anticipated that the range of unallotted exit codes will be further restricted in the future. The 
author of this document will not do fixups on the scripting examples to conform to the changing 
standard. This should not cause any problems, since there is no overlap or conflict in usage of exit codes 


between compiled C/C++ binaries and shell scripts. 
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Appendix E. A Detailed Introduction to I/O and I/O 
Redirection 


written by Stéphane Chazelas, and revised by the document author 


A command expects the first three file descriptors to be available. The first, fd O (standard input, st din), is 
for reading. The other two (fd 1, stdout and fd 2, stderr) are for writing. 


There is a stdin, stdout, and a stderr associated with each command. 1s 2»&1 means temporarily 
connecting the stderr of the Is command to the same "resource" as the shell's stdout. 


By convention, a command reads its input from fd 0 (st din), prints normal output to fd 1 (stdout), and 
error ouput to fd 2 (stderr). If one of those three fd's is not open, you may encounter problems: 


bash$ cat /etc/passwd »&- 
cat: standard output: Bad file descriptor 


For example, when xterm runs, it first initializes itself. Before running the user's shell, xterm opens the 
terminal device (/dev/pts/«n» or something similar) three times. 


At this point, Bash inherits these three file descriptors, and each command (child process) run by Bash inherits 
them in turn, except when you redirect the command. Redirection means reassigning one of the file 
descriptors to another file (or a pipe, or anything permissible). File descriptors may be reassigned locally (for 
a command, a command group, a subshell, a while or if or case or for loop...), or globally, for the remainder of 
the shell (using exec). 


ls > /dev/null means running Is with its fd 1 connected to /dev/nul11. 


bash$ lsof -a -p $$ -d0,1,2 


COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 

bash 363 bozo Ou CE — 136,1 3 /dev/pts/1 
bash 33 looo lu CHR SG 3 /dev/pts/1 
bash Sieg) Toxoya(o) 2u Chir 136, 3 /dev/pts/1 


bash$ exec 2» /dev/null 
bash$ lsof -a -p $$ -d0,1,2 


COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 

bash STL OOA Ou CE ISG, i 3 /dev/pts/1 
bash 371 DOZO iil CHR 136.1 3 /dev/pts/1 
bash STL BOZO 2w CHR pS 120 /dev/null 


bash$ bash -c 'lsof -a -p $$ -d0,1,2' | cat 


COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 
lesot 379 root Ou CHR 136,1 3 /dev/pts/1 
lsof STS woo lw FIFO 0, (0) 7118 pipe 
liget 379 root 2u CHR 136,1 3 /dev/pts/1 


bash$ echo "$(bash -c 'lsof -a -p $$ -d0,1,2' 2>&1)" 
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 


lsof 426 root Ou CHR — 3,9085 1i 3 /dev/pts/1 
leot 426 root lw FIFO O, (9) 7520 pipe 
lsof 426 root 2w FIFO @, (9) 7520 pipe 


This works for different types of redirection. 


Exercise: Analyze the following script. 


#! /usr/bin/env bash 


Wnkftslito) Jemo iiio Jiu [3o 
waile reacli ep clo echo "39s Sap done < /iug/ixirgl & esee T> /upjs:suiitol 
exec B (wlule reac ap clo echo “WDE: Sa, To Fole cons >67) 


exec 3>&1 
( 
( 
( 
wile reac cip elo) Scho WigmO2s3 Sale clone < /icime/icuico® | tes clenr/Siecleeie SN 
| tee /dev/fd/4 | tee /dev/fd/5 | tee /dev/fd/6 >&7 & exec 3» /tmp/fifo2 


NPRPRPRP PPP PP 
COMIKDGTKRWNHFOWOCMIDUARWNE 


echo lst, to stdout 
sleep 1 
echo 2nd, to stderr >&2 
sleep 1 
echo See to fd 3 55 
sleep 1 
echo 4th, to fd 4 364 
2l sleep 1 
D echo 5th, to fd 5 55 
29 Sleep 1 
24 echo Gth, throch a pipe | sed Ys/. PUPS &, EO EC 5/" 9&5 
25 sleep 1 
26 echo 7th, to fd 6 456 
27 Sleep 1 
28 echo th, to fd 7 267 
29 sleep 1 
30 echo Sth, to fa 9 245 
Sil 
3e ) 4>&1 >&3 3>&- | while read a; do echo "FD4: $a"; done 1>&3 5>&- 6>&- 
33 J Sse Ses | wails reac ap clo echo UiBIDSE Sale ke l>e G>% 
34 ) 6>&1 >&3 | while read a; do echo "FD6: Sa"; done 3»&- 
25 
36 MM -E /oma/fiiol J/upo/i1£92 
37 
38 


39 # For each command and subshell, figure out which fd points to what. 
40 # Good luck! 


41 

42 exit 0 
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Appendix F. Command-Line Options 


Many executables, whether binaries or script files, accept options to modify their run-time behavior. For 
example: from the command-line, typing command -o would invoke command, with option o. 


F.1. Standard Command-Line Options 


Over time, there has evolved a loose standard for the meanings of command-line option flags. The GNU 
utilities conform more closely to this "standard" than older UNIX utilities. 


Traditionally, UNIX command-line options consist of a dash, followed by one or more lowercase letters. The 
GNU utilities added a double-dash, followed by a complete word or compound word. 


The two most widely-accepted options are: 
e -h 
--help 


Help: Give usage message and exit. 
e-v 


—-version 
Version: Show program version and exit. 
Other common options are: 
e-a 
--all 


All: show all information or operate on all arguments. 
e-1 


--list 


List: list files or arguments without taking other action. 
e -o 


Output filename 
-q 


--quiet 


Quiet: suppress stdout. 
e-r 


-R 
--recursive 


Recursive: Operate recursively (down directory tree). 
eV 


--verbose 


Verbose: output additional information to stdout or stderr. 


227 
--compress 
Compress: apply compression (usually gzip). 
However: 
* [n tar and gawk: 
-f 
--file 


File: filename follows. 
* In cp, mv, rm: 


-—force 
Force: force overwrite of target file(s). 


Many UNIX and Linux utilities deviate from this "standard," so it is dangerous to assume that a given 
option will behave in a standard way. Always check the man page for the command in question when in 
doubt. 


A complete table of recommended options for the GNU utilities is available at the GNU standards page. 
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F.2. Bash Command-Line Options 


Bash itself has a number of command-line options. Here are some of the more useful ones. 
.-c 


Read commands from the following string and assign any arguments to the positional parameters. 


bash$ bash -c 'set a b c d; IFS="+-;"; echo "$*"' 
atb+ct+d 
e-r 
--restricted 


Runs the shell, or a script, in restricted mode, 
e -—posix 


Forces Bash to conform to POSIX mode. 
e -—version 


Display Bash version information and exit. 
e —— 


End of options. Anything further on the command line is an argument, not an option. 
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Appendix G. Important Files 


startup files 
These files contain the aliases and environmental variables made available to Bash running as a user 
shell and to all Bash scripts invoked after system initialization. 

/etc/profile 
Systemwide defaults, mostly setting the environment (all Bourne-type shells, not just Bash [1]) 

/etc/bashrc 
systemwide functions and aliases for Bash 

SHOME/.bash, profile 
user-specific Bash environmental default settings, found in each user's home directory (the local 
counterpart to /etc/profile) 

SHOME/.bashrc 
user-specific Bash init file, found in each user's home directory (the local counterpart to 
/etc/bashrc). Only interactive shells and user scripts read this file. See Appendix L for a sample 
.bashrc file. 


logout file 


SHOME/.bash, logout 
user-specific instruction file, found in each user's home directory. Upon exit from a login (Bash) shell, 
the commands in this file execute. 


data files 

/etc/passwd 
A listing of all the user accounts on the system, their identities, their home directories, the groups they 
belong to, and their default shell. Note that the user passwords are not stored in this file, [2] but in 


/ etc/shadow in encrypted form. 


system configuration files 


/etc/sysconfig/hwconf 
Listing and description of attached hardware devices. This information is in text form and can be 
extracted and parsed. 


bash$ grep -A 5 AUDIO /etc/sysconfig/hwconf 

class: AUDIO 

louis ICI 

detached: 0 

driver: snd-intel8x0 

desc: "Intel Corporation 82801CA/CAM AC'97 Audio Controller" 
vendorId: 8086 


$") This file is present on Red Hat and Fedora Core installations, but may be missing from 
other distros. 


Notes 


[1] This does not apply to csh, tcsh, and other shells not related to or descended from the classic Bourne 
shell (sh). 


[2] In older versions of UNIX, passwords were stored in /et c/passwd, and that explains the name of the 
file. 
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Appendix H. Important System Directories 


Sysadmins and anyone else writing administrative scripts should be intimately familiar with the following 
system directories. 


e /bin 


Binaries (executables). Basic system programs and utilities (such as bash). 
e/usr/bin[ll 


More system binaries. 
e /usr/local/bin 


Miscellaneous binaries local to the particular machine. 
e /sbin 


System binaries. Basic system administrative programs and utilities (such as fsck). 
e /usr/sbin 


More system administrative programs and utilities. 
e/etc 


Et cetera. Systemwide configuration scripts. 


Of particular interest are the /etc/ fstab (filesystem table), /et c/mt ab (mounted filesystem 


table), and the /etc/inittab files. 
e /ctc/rc.d 


Boot scripts, on Red Hat and derivative distributions of Linux. 
e /usr/share/doc 


Documentation for installed packages. 
e/usr/man 


The systemwide manpages. 
e /dev 


Device directory. Entries (but not mount points) for physical and virtual devices. See Chapter 29. 
e /proc 


Process directory. Contains information and statistics about running processes and kernel parameters. 


See Chapter 29. 
e /sys 


Systemwide device directory. Contains information and statistics about device and device names. This 
is newly added to Linux with the 2.6.X kernels. 
e /mnt 


Mount. Directory for mounting hard drive partitions, such as /mnt / dos, and physical devices. In 
newer Linux distros, the /media directory has taken over as the preferred mount point for I/O 
devices. 

e /media 


In newer Linux distros, the preferred mount point for I/O devices, such as CD/DVD drives or USB 
flash drives. 
e /var 


Variable (changeable) system files. This is a catchall "scratchpad" directory for data generated while 
a Linux/UNIX machine is running. 


e /var/log 


Systemwide log files. 
e /var/spool/mail 


User mail spool. 
e/lib 


Systemwide library files. 
e /usr/lib 


More systemwide library files. 
e /tmp 


System temporary files. 
e /boot 


System boot directory. The kernel, module links, system map, and boot manager reside here. 


Ed Altering files in this directory may result in an unbootable system. 


Notes 


[1] Some early UNIX systems had a fast, small-capacity fixed disk (containing /, the root partition), and a 
second drive which was larger, but slower (containing /usr and other partitions). The most frequently 
used programs and utilities therefore resided on the small-but-fast drive, in /bin, and the others on the 
slower drive, in /usr/bin. 


This likewise accounts for the split between / sbin and /usr/sbin,/liband /usr/lib,etc. 
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Appendix I. An Introduction to Programmable 
Completion 


The programmable completion feature in Bash permits typing a partial command, then pressing the [Tab] key 
to auto-complete the command sequence. [1] If multiple completions are possible, then [Tab] lists them all. 
Let's see how it works. 


bash$ xtra[Tab] 


Xtraceroute xtrapin xtrapproto 
xtraceroute.real xtrapinfo xtrapreset 
xtrapchar XLEapout xtrapstats 


bash$ xtrac[Tab] 
xtraceroute xtraceroute.real 


bash$ xtraceroute.r[Tab] 
xtraceroute.real 


Tab completion also works for variables and path names. 


bash$ echo $BASH [Tab] 


SBASH S$BASH COMPLETION S$BASH SUBSHELL 
SBASH ARGC S$BASH COMPLETION DIR  $BASH VERSINFO 
S$BASH ARGV S$BASH LINENO S$BASH VERSION 
SBASH COMMAND SBASH SOURCE 


bash$ echo /usr/local/ [Tab] 
bin/ etc/ include/ libexec/ sbin/ eui, 
doc/ games / lib/ man/ share/ 


The Bash complete and compgen builtins make it possible for tab completion to recognize partial parameters 
and options to commands. In a very simple case, we can use complete from the command-line to specify a 
short list of acceptable parameters. 


bash$ touch sample command 

bash$ touch filel.txt file2.txt file2.doc file30.txt file4.zzz 
bash$ chmod +x sample command 

bash$ complete -f -X '!*.txt' sample command 


bash$ ./sample [Tab] [Tab] 
sample command 
ical del ese, ELIGA esac iz 31 1L: edes: 


The -f option to complete specifies filenames, and -X the filter pattern. 


For anything more complex, we could write a script that specifies a list of acceptable command-line 
parameters. The compgen builtin expands a list of arguments to generate completion matches. 


Let us take a modified version of the UseGetOpt.sh script as an example command. This script accepts a 
number of command-line parameters, preceded by either a single or double dash. And here is the 
corresponding completion script, by convention given a filename corresponding to its associated command. 


Example I-1. Completion script for UseGetOpt.sh 


1 # file: UseGetOpt-2 
2 4 UseGetOpt-2.sh parameter-completion 
3 
4  UseGetOpt-2 () # By convention, the function name 
S 4 #+ starts with an underscore. 
6 Toca Meur 
7 # Pointer to current completion word. 
8 # By convention, it's named "cur" but this isn't strictly necessary. 
9 
10 COMPREPLY-() # Array variable storing the possible completions. 
i cur=S$ {COMP_WORDS [COMP_CWORD] } 
12 
13 case “Seuss? aia 
14 =) 
15 COMPREPLY=( S eueupgsea i -a -Cl =i -1 =e h aoption --debug \ 
16 Eii log test help i Sew ) ee 
17 # Generate the completion matches and load them into $COMPREPLY array. 
18 # xx) May add more cases here. 
19 # yy) 
20 # ZZ) 
21 esae 
27 
29 return 0 
24 ) 
25 


26 complet F _UseGetOpt-2 -o filenames ./UseGetOpt-2.sh 
27 t (sew Ke sISONCSUNGMANON NS UN WORSE Elm Unc tiOn . WSSESECOIE=Z . 


Now, let's try it. 


bash$ source UseGetOpt-2 


bash$ ./UseGetOpt-2.sh -[Tab] 
== --aoption  --debug ies. JL help ==llog T test 
-a =ë] =É =h =1 -t 


bash$ ./UseGetOpt-2.sh --[Tab] 
== --aoption  --debug Ean help = leg --test 


We begin by sourcing the "completion script." This sets the command-line parameters. [2] 


In the first instance, hitting [Tab] after a single dash, the output is all the possible parameters preceded by one 
or more dashes. Hitting [Tab] after two dashes gives the possible parameters preceded by two or more dashes. 


Now, just what is the point of having to jump through flaming hoops to enable command-line tab completion? 
It saves keystrokes. [3] 


Resources: 
Bash programmable completion project 
Mitch Frazier's Linux Journal article, More on Using the Bash Complete Command 


Steve's excellent two-part article, "An Introduction to Bash Completion": Part 1 and Part 2 


Notes 


[1] This works only from the command line, of course, and not within a script. 

[2] Normally the default parameter completion files reside in either the /etc/profile.d directory or in 
/etc/bash completion. These autoload on system startup. So, after writing a useful completion 
script, you might wish to move it (as root, of course) to one of these directories. 

[3] Ithas been extensively documented that programmers are willing to put in long hours of effort in order 
to save ten minutes of "unnecessary" labor. This is known as optimization. 
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Appendix J. Localization 


Localization is an undocumented Bash feature. 


A localized shell script echoes its text output in the language defined as the system's locale. A Linux user in 
Berlin, Germany, would get script output in German, whereas his cousin in Berlin, Maryland, would get 
output from the same script in English. 


To create a localized script, use the following template to write all messages to the user (error messages, 
prompts, etc.). 


#!/bin/bash 

# localized.sh 

# Script by Stéphane Chazelas, 

#+ modified by Bruno Haible, bugfixed by Alfredo Pironti. 


gettext.sh 


E CDERROR-65 


error() 

( 
joueahowege: WS mE 
exit $E CDERROR 


Mop pop opp PPP PY 
O (o 0» -1 O Oi i$ (). NM P. O xo 0 -1 O O1 4 CQ Io E 


ecl Swaie [|] error eyel cercani Can "ie col io \\\Swae. VU 
The triple backslashes (escapes) in front of $var needed 
+ "becaus val gettext expects a string 
+ where the variable values have not yet been substituted." 
-- per Bruno Haible 

Di wail 49 V certe Vinton iae values W U xe 
22 
23 
24 
25 
26 Alfredo Pironti comments: 
2 
28 This script has been modified to not use the $"..." syntax in 
20) v Seeley (our jelavey VW Gere WY 55 aN SAEC 
30 This is ok, but with the new localized.sh program, the commands 
31 #+ "bash -D filename" and "bash --dump-po-string filename" 
32 #+ will produce no output 
33 #+ (because those command are only searching for the $"..." strings)! 
34 The ONLY way to extract strings from the new file is to use the 
35 'xgettext' program. However, the xgettext program is buggy. 
36 
97 Note that 'xgettext' has another bug. 
38 
39 The shell fragment: 
40 gettext -s "I like Bash" 
Ad) will be correctly extracted, but 
42 xgettext -s "I like Bash" 
43 ite dbrss 
44 'Xgettext' will extract "-s" because 
45 #+ the command only extracts the 
46 #+ very first argument after the 'gettext' word. 
47 
48 
49 Escape characters: 
50 
Sil To localize a sentence lik 
52 echo -e "Hello\tworld!" 


53 #+ you must use 

54 cho U geltet \Wisieililo\ yenor llchun u 

55 The "double escape character" before the `òt' is needed because 
56 #+ 'gettext' will search for a string like: 'HelloNtworld' 
$7 This is because gettext will read one literal ~\') 

58 #+ and will output a string like "Bonjour\tmonde", 

59 #+ so the 'echo' command will display the message correctly. 
60 

61 You may not use 

62 echo "gettext NISI dL le Nes JL NW Y 

63 #+ due to the xgettext bug explained abov 

64 

65 

66 

67 Let's localize the following shell fragment: 

68 echo "-h display help and exit" 

69 

70 PileSt, que uou clo ims 

qi echot gettext \Y ala chisjsileny leds  uevel exalt VU UU 

72 This way 'xgettext' will work ok, 

73 #+ but the 'gettext' program will read "-h" as an option! 

74 

TS One solution could be 

76 echo "gettext -- \"-h display help and ezic Y 

a This way 'gettext' will work, 

78 #+ but 'xgettext' will extract "--", as referred to above. 
p 

80 The workaround you may use to get this string localized is 
81 cho "gettext \"\\0-h display help and exitV"^" 

82 We have added a \0 (NULL) at the beginning of the sentence. 
83 This way 'gettext' works correctly, as does 'xgettext.' 
84 Moreover, the NULL character won't change the behavior 

85 #+ of the 'echo' command. 

86 


bash$ bash -D localized.sh 
Uu i, Gel ico) Wee" 
"Enter the value: " 


This lists all the localized text. (The —D option lists double-quoted strings prefixed by a $, without executing 
the script.) 


bash$ bash --dump-po-strings localized.sh 
WB es 

Insgs'ceu eom CCl CO g3% 

Ine erster Wy 

f: a:7 

msgid "Enter the value: " 

msgstr UU 


The --dump-po-strings option to Bash resembles the -D option, but uses gettext "po" format. 


$^; Bruno Haible points out: 


Starting with gettext-0.12.2, xgettext -o - localized.sh is recommended instead of bash 
--dump-po-strings localized.sh, because xgettext . . . 


1. understands the gettext and eval_gettext commands (whereas bash --dump-po-strings understands only 
its deprecated $"..." syntax) 


2. can extract comments placed by the programmer, intended to be read by the translator. 


This shell code is then not specific to Bash any more; it works the same way with Bash 1.x and other 
/bin/sh implementations. 


Now, build a language. po file for each language that the script will be translated into, specifying the 
msgstr. Alfredo Pironti gives the following example: 


fr.po: 
L 43 286 
2 megicl “Cam: ec to Swarr. 
3 msgstr "Impossible de se positionner dans le repertoire $var." 
4 #: a:7 
5 msgid "Enter the value: " 
(5 megsti "durer la valeur g V 
7 
8 # The string are dumped with the variable names, not with the %s syntax, 
9 #+ similar to C programs. 
10 #+ This is a very cool feature if the programmer uses 
11 #+ variable names that make sense! 


Then, run msgfmt. 


msgfmt -o localized.sh.mo fr.po 


Place the resulting 1ocalized.sh.mo file in the /usr/local/share/locale/fr/LC, MESSAGE 
directory, and at the beginning of the script, insert the lines: 


[02] 


1 TEXTDOMAINDIR-/usr/local/share/locale 
2 TEXTDOMAIN=localized.sh 


If a user on a French system runs the script, she will get French messages. 


$^; With older versions of Bash or other shells, localization requires gettext, using the -s option. In this 
case, the script becomes: 


#!/bin/bash 
# localized.sh 


__CDERROR=65 


error() { 
local format=$1 
SMi iE qe 
primet "S(gettew: -s VSformat) Y "Sp" See 
10 exit $E CDERROR 
Jom 
112 exl svaz ||| error "Cau" col to Ss." Vevar” 
l3 reacli -p YS (gettext -3 Viae (tbe value: 7) var 
14 # 


The TEXTDOMAIN and TEXTDOMAINDIR variables need to be set and exported to the environment. This 
should be done within the script itself. 


© ~] oy, OV d 69 IND ES 


ie) 


This appendix written by Stéphane Chazelas, with modifications suggested by Alfredo Pironti, and by Bruno 
Haible, maintainer of GNU gettext. 
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Appendix K. History Commands 


The Bash shell provides command-line tools for editing and manipulating a user's command history. This is 
primarily a convenience, a means of saving keystrokes. 


Bash history commands: 


1. history 
2. fc 


bash$ history 
1 mount /mnt/cdrom 
2 eel uns celum 
3 Is 


Internal variables associated with Bash history commands: 


1. SHISTCMD 
2. SHISTCONTROL 
3. SHISTIGNORE 
4. $HISTFILE 
5. SHISTFILESIZE 
6. SHISTSIZE 
7. SHISTTIMEFORMAT (Bash, ver. 3.0 or later) 
8. !! 
9. !$ 
10. !# 
11.!N 
12. !-N 
13. STRING 
14. !9STRING? 
15. ^STRING^string^ 


Unfortunately, the Bash history tools find no use in scripting. 


1 #!/bin/bash 

2 a; Imole slm 

3 4 A (vain) attempt to use the 'history' command in a script. 
4 

5 history # No output. 

6 

7 var-$(history); echo "Svar" # Svar is empty. 

8 

9 # History commands disabled within a script. 


bash$ ./history.sh 
(no output) 


The Advancing in the Bash Shell site gives a good introduction to the use of history commands in Bash. 
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Appendix L. A Sample .bashrcFile 


The ~/.bashrc file determines the behavior of interactive shells. A good look at this file can lead to a 
better understanding of Bash. 


Emmanuel Rouat contributed the following very elaborate . bashrc file, written for a Linux system. He 
welcomes reader feedback on it. 


Study the file carefully, and feel free to reuse code snippets and functions from it in your own .bashrc file 
or even in your scripts. 


Example L-1. Sample .bashrc file 


as) 


ERSONAL SHOME/.bashre FILE for bash-3.0 (or later) 
By Emmanuel Rouat <no-email> 


Last modified: Sun Nov 30 16:27:45 CET 2008 
This file is read (normally) by interactive shells only. 
Here is the place to define your aliases, functions and 
other interactive features like your prompt. 


The majority of the code here assumes you are on a GNU 
system (most likely a Linux box) and is based on code found 
on Usenet or internet. See for instance: 


http://tldp.org/LDP/abs/html/index.html 
http://www.caliban.org/bash/ 
http://www.shelldorado.com/scripts/categories.html 
http://www.dotfiles.org/ 


NPRPRPRPRP PPP PY 
O io 00 -1 OY O1 i& (0. NM HIS. O xo 0 -1 O O1 4S CQ I E 


This bashrc file is a bit overcrowded remember it is just 


2L just an example. Tailor it to your needs. 

22 

253 

24 

25 

26 --» Comments added by HOWTO author. 

Bl 

28 

29 

30 Source global definitions (if any) 

ES 

32 

33 

34 if [ -f /etc/bashre ]; then 

35 /etc/bashrc # --» Read /etc/bashrc, if present. 
SG ral 

37 

38 

39 Automatic setting of SDISPLAY (if not set already). 
40 This works for linux - your mileage may vary. 

41 The problem is that different types of terminals give 
42 different answers to 'who am i' (rxvt in particular can be 
43 troublesome). 

44 I have not found a 'universal' method yet. 

45 

46 


47 function get xserver () 
48 ( 


49 
50 
Sil 
52 
59 
54 
55 
56 
S7 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
al 
12 
73 
74 
1/5 
76 
T9 
78 
HS 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
2d 
22 
9S 
94 
95 
96 
2 
98 
29 
100 
101 
102 
103 
104 
105 
106 
107 
108 
109 
110 
111 
1312 
S 
114 


case STERM in 
xterm ) 


esac 


x 
ca 
"d 
< 


R=S (ioe) am sh || awk "exea. Sie}! || wie l Vy vv ) 
Ane-Pieter Wieringa suggests the following alternative: 
I_AM=S (who am i) 

SERVER=S { I_AM#* (} 

SERVER=S { SERVER%*) } 


I 


XSERV. 


R=$ {XSERVERS3: *} 


aterm | rxvt) 
# Find some code that works here. 


vr 


if [ -z S(DISPLAY:-"") ]; then 
got server 


TRAE 


el 


ít 
ita 


export 


[ 
$t 


SẸ 


D 


[ -z ${XSERVER} DiS EXSERVER |= ——so( hostname) ill iien\ 
XSERVER} == "unix" ]]; then 
DISPLAY=":0.0" Display on local host. 


DISPLAY=S$ {XSERVER}:0.0 Display on remote host. 


TSP LAY 


# 
# Some 
# 


S 


ettings 


ulimit 
BEE =O 
Soto 
Siren 
GEE t 


eese (0) # Don't want any coredumps. 
notify 


n 
i 
n 


oclobber 
gnoreeof 
ounset 


#set -o xtrace # Useful for debuging. 


Enab 
hopt 
hopt 
hopt 
hopt 
hopt 
hopt 
hopt 
hopt 
hopt 


tou qn. fen pvo (ur (v [n dor c 


le 


> 
= 


options: 

cdspell 

cdable_vars 

checkhash 

checkwinsize 

sourcepath 

no_empty_cmd_completion 

emdhist 

histappend histreedit histverify 

extglob # Necessary for programmable completion. 


# Disable options: 
shopt -u mailwarn 


unset MAILCHECK # Don't want my shell to warn me of incoming mail. 
export TIMEFORMAT=$'\nreal %3R\tuser $3UNtsys %3S\tpcpu %P\n' 

export HISTTIMEFORMAT="%H:%M > " 

export HISTIGNORE-2"&:bg:fg:ll:h" 

export HOSTFILE-S$HOME/.hosts f Put list of remote hosts in -/.hosts 

# 


# Greeting, motd etc... 


# 


LAL) 
116 
1:1) 
118 
119 
120 
11271 
122 
12:3) 
124 
1,255) 
126 
127 
128 
129 
130 
JL SL 
132 
193 
134 
135 
136 
137, 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
15i 
152 
153 
154 
15:5) 
156 
1.5/7) 
1.515 
15$ 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
173 
dL 02 
NS) 
174 
S 
1L 78. 
37.3 
178 
LHS) 
180 


# Define some colors first: 
red='\e[0;31m' 


RED='\e[1;31m' 


blue='\e[0;34m' 
BLUE-'Ne[1;34m' 
cyan='\e[0;36m' 
CAGANEY We [[ 1L 8 Shem" 


NC wes rou" 
# --» Nice. 


# No Color 


Has the same effect as using "ansi.sys" 


# Looks best on a terminal with black background... 
echo -e "S(CYAN)This is BASH ${RED}${BASH_VERSION3.*}\ 
S{CYAN} - DISPLAY on ${RED}S$DISPLAYS{NC}\n" 


date 


if [ -x /usr/games/fortune ]; then 


/usr/games/fortune -s 


1E3L 


ware ake —eXxabie (()} 


{ 


echo -e 


) 


(EJOEIS. eS 


iia DOSE 


# Makes our day a bit more fun.... 


# Function to run upon exit of shell. 


"S{RED}Hasta la vista, baby${NC}" 


EXIT 


# 
# Shell Prompt 
# 
ade [PP "SEE gg I WW ie Eee 
HILIT-$(red) # remote machine: prompt will be partly red 
else 
HILIT-$(cyan) # local machine: prompt will be partly cyan 
fi 
#  --» Replace instances of \W with Nw in prompt functions below 


#+ --» to get display of full path name. 


function fastprompt () 


{ 


T}[\h]$NC NW > N[N033]0; NS(T 


DE Wat] SIG wo w sh 


unset PROMPT COMMAND 
case STERM in 
aeea | xv) 
PS1="${HILI1 
Tue )) 
PS1="${HILI1 
A, 
PS1="[\ 


esac 


_powerprompt () 


{ 


LOAD-$ (uptime|sed -e "s/.*: 


function powerprompt () 


{ 


aj Www » "V ss 


PROMPT COMMAND- powerprompt 


case STI 


ERM in 


‘mera || xev D) 


PSI-"S(HILIT)[NA — \S$LOAD]SNC\n[\u@\h NK] 


ERM} 


[\u@\h] 


X(L^g]^N)29/7NILZ9 =e Ve Fife") 


\w > \ 


NwN007N]" 


, 


, 


181 NI[IN033]0; NS (TERM). [\u@\h] NwNOO7NT" ;; 

182 linux ) 

1183) PS1="S{HILIT}[\A - \SLOAD]SNC\n[\u@\h NE] NW >" ;; 
184 K 

185 PSi=" [VA = NORD Wal Wwe vin Wil WW * " gg 

186 ese 

i97 

188 

189 powerprompt # This is the default prompt -- might be slow. 
190 # If too slow, use fastprompt instead. 

1. 93L 
192 
JL 913) 
194 ALIASES AND FUNCTIONS 
T95 
196 Arguably, some functions defined here are quite big. 

LOT If you want to make this file smaller, these functions can 
198 be converted into scripts and removed from here. 

LOS) 
200 Many functions were taken (almost) straight from the bash-2.04 
20i examples. 

202 
203 
204 
205 
206 Personnal Aliases 
207 
208 
209 alias rm='rm -i' 


210 alias Gom Go 3t 

211 alias mv='mv -i' 

212 4 -» Prevents accidentally clobbering files. 

As) alias nnui meihe 9" 

214 

215 alias h-'history' 

2JL5 gullies qe" ees =i! 

217 alias which='type -a' 

2L. gillateis: 4 oc eel ns 

219 alias path-'echo -e ${PATH//:/\\n}' 

220 alias libpath-'echo -e $(LD LIBRARY PATH//:/NNnJ' 

221 alias print='/usr/bin/lp -o nobanner -d SLPDEST' 

22 # Assumes LPDEST is defined (default printer) 

223 alias pjet-'enscript -h -G -fCourier9 -d SLPDEST' 

224 # Pretty-print using enscript 

2285) 

22/9 ekas clic =n # Makes a more readable output. 

227 mates. Che he lela 

228 

229 # 

230 # The 'ls' family (this assumes you use a recent GNU 1s) 

231 # 

Doe eias ISVs; E group-directories-first" 

233 Alias Iss" iis: Ju (ele add colors for filetype recognition 
234 alias la='ls -AI' show hidden files 

2.95. mates. Js) ls; IDI sort by extension 

29/6 Gulaas JUkevdie ues sort by size, biggest last 

29:7 eieae leder deese sort by and show change time, most recent last 
238 alias lu2'ls -ltur' sort by and show access time, most recent last 
239 alias lt-'ls —-ltr' Sort by date, most recent last 

240 alias lm-'ls -al |more' pipe through 'more' 

Zh eue Jc dier ERA recursive ls 

242 alias tree-'tree -Csu' nice alternative to 'recursive l1s' 
243 

244 # If your version of 'ls' doesn't support group-directories-first try this: 
DAS s iebincicaloim JL1L() 4 ls sil “Sel | egrad "^s" p Te -IKS MSE" 2»-&—| \ 


246 # ens —w WeglliguesdL Ws } 


247 
248 
249 
250 
25l 
252 
253 
254 
255 
256 
257 
258 
299) 
260 
261 
262 
263 
264 
265 
266 
267 
268 
269 
270 
Ziel 
272 
DAS 
274 
275 
276 
2-03) 
278 
279 
280 
281 
282 
283 
284 
285 
286 
287 
288 
289 
290 
291 
ADD 
293 
294 
295 
296 
297 
298 
299 
300 
301 
302 
303 
304 
305 
306 
307 
308 
309 
SLO 
Sib 
SZ 


# 
# tailoring 'less' 
# 
alias more='less' 


export PAGER=less 
export LESSCHARSET-'latinl' 
export LESSOPEN-'|/usr/bin/lesspipe.sh $s 2>&-' 
4 Use this if lesspipe.sh exists 
export LESS-'-i -N -w zu g M =< sf SR ie Y 


GsicCliin . Voosslo\sePWomaines Silly Ploloshyice kalelo qo ooo ! 
# 
# spelling typos - highly personnal and keyboard-dependent 


alias xs='cd' 
alias vf='cd' 
alias moer='more' 
alias moew='more' 
alias kk-'ll' 


# 
# A few fun ones 
# 
UMC som scale Te) # Adds some text in the terminal frame. 
{ 
case "STERM" in 
*term | rxvt) 
Scho -n =o OSSOS \OOT" PF 
ho, 
Pr; 
esac 
} 
# aliases that use xtitle 
alias top='xtitle Processes on $HOST && top' 
alias make-'xtitle Making $(basename $PWD) ; make' 
alias mentorice Yel" p. eE 
# .. and functions 
function man() 
( 
BORS aL £ oko) 
xtitle The $(basename $1|tr -d .[:digit:]) manual 
Counmmeaiayel men = el UU 
done 
} 
# 
# Make the following commands run in background automatically: 
# 
function te() # Wrapper around xemacs/gnuserv 
{ 
ae [p VS (cGaveliecmnt barch -eval ic 2245—) Y == Wis! Hg. then 
gaceliene =e MSE"? 
else 
( xemaes Um" m) 


31.3] 
314 
SIL) 
316 
317) 
318 
319 
320 
321 
322 
3213) 
324 
325) 
326 
327) 
328 
329) 
330 
S3 
S32 
3.3.3) 
334 
335 
336 
337 
IS 
33 
340 
341 
342 
343 
344 
345 
346 
347 
348 
349 
390 
Soll 
352 
3S3) 
354 
395 
356 
357 
S965 
S99 
360 
361 
362 
SIS 
364 
365 
366 
367 
368 
369 
310 
Sn 
372 
uS 
374 
S75) 
376 
377 
378 


ita 


fanet dua Sorrtlee() { command Sorties "SU m } 
function AcsLiessceo<() X Cem! Pireo WSAY m p 
function xpdf() { command xpdf "S@" & } 


File & string-related functions: 


Find a file with a pattern in name: 
iftius E3Loum ii (()) “| itamel , Sees it —meue Yoo l Sve? ole p D 


Find a file with pattern $1 in name and Execute $2 on it: 
function fe() 
( dE3Gmel o —itwqe 1 Simms "SIE. see sies fi we 


# Find a pattern in a set of files and highlight them: 
# (needs a recent version of egrep) 
EUNET skola ESEE) 


{ 


OPTIND=1 
local casez"" 
local usage="fstr: find string in files. 
Usiagec terci il \Wisercicerem\Y [WU sei eines. joaieiceica\ Y 
while getopts :it opt 


do 
case "Sopt" in 
30 Gies W 3 
*) echo "Susage"; return;; 
esac 
done 
siad GC SOLD = i p») 
ise [p Uwe sili dL qo ilex 
echo "$usage" 
return; 
i3 


sme o Eye de nein "US25—9]U orim. || \ 


, 


xargs -0 egrep color-always -sn $(case) "$1" 2>&- | more 


function cuttail() # cut last n lines in file, 10 by default 


( 
nlines=${2:-10} 
sed -n ta WL, SiAiallaines } HP os DA P elelea" Sl 


function lowercase () # move filenames to lowercase 
{ 
itíoue iIlS 9 Cle) 
filename=S{file##*/} 
case "Sfilename" in 
afa quemewas--BS4icilee/*)) Bp 
*) dirname-.;; 
esac 
nf=$ (echo $filename | tr A-Z a-z) 
newname-"$(dirname)/S$ínf)" 


me [p Wisin dies Uer icseuns Tp elem 

mv "Sfile" "$newname" 

echo "lowercase: $fil > Snewname" 
else 


echo "lowercase: $file not changed." 


43 


done 


function swap() 


{ 


local TMPETI! 


# Swap 2 filenames around, 
# (from Uzi's bashrc). 
E-tmp.$$ 


if they exist 


&& return 1 
&& return 1 
&& return 1 


[ $4 -ne 2 ] && echo "swap: 2 arguments needed" 
[ ! -e $1] && echo "swap: $1 does not exist" 
[ ! -e $2 ] && echo "swap: $2 does not exist" 
wm WELW GENRET Lia 
my UAY Uen 
mv STMPFILE "S2" 
} 
function extract () # Handy Extract Program. 
{ 
aise [[ Sad Sal] 5o AEA 
case $1 in 
aea olov ) ieee’ cw deu RE 
TS 6 Erbe S 6%) ineuc Svr Sil B 
D z2023) bunzip2 $1 TET 
RA) toene 5x Sil, RE 
"SS GE) gunzip $1 ET 
+ Ie eue)) ieee xewi Sal EE 
hcl son tar xvjf $1 $3 
AZA) ipee. Sev Sl as 
WSLS) vazig Si BE 
io A) uncompress $1 BE 
AEZ) Ju x mul 8g 
*) fexolayoy uU ssi 
esac 
else 
excimo) Wielt as Dor xb valie sen ike 
fa 
} 
# 
# Process/system related functions: 
# 
EMMIGELOIN mys) 1 is SC =u SUS 


function pp() 


PUNE EONI KAOS) 


{ 


local pid pname sig="-TERM" 
ales AL Je qni 
killps 


as qp wes 
echo "Usage: 
return; 

1E 3b 

x [| Sy = 2 


itose joel nim S (amy 9e 
pname-$ (my ps | 


] 


( my ps f | 


awk 


!'!l/awk/ && SO~var' 


renes lip ces 


# Kill by process name. 


"SH" -gt 2 
[-SIGNAL] 


G deloxewct aber 9 esL 


awk 
awk 


]; then 
pattern" 


# Default signal. 


if ask "Kill process $pid <Spname> with signal $sig?" 
then kill $sig $pid 


done 


44 function my ip() 


# Get IP adresses. 


ER -o pid,$cpu,$mem,bsdtime,command ; 


) 


'l/awk/ && SO~pat ( print $1 )' pat=S{!#} 
/SdLcswsue. di porine (us. jh" wwene=Sjoyicl ) 


cannot be extracted via >extract<" 


} 


Mi =S sbin ni reont jsyos || ewik "iue 4 prime $92 Jj V || X 
sed -e s/addr://) 
MY ISP-$(/sbin/ifconfig ppp0 | awk '/P-t-P/ { print $3 ) ' | 
sed -e s/P-t-P://) 
} 
PASE aLeya, 3bsb (()) # Get current host related info. 
{ 
echo =e "\nYou are logged on S$(RED)SHOST" 
echo e "XnAdditionnal information:$NC " A uname -a 
acho SS HAns (RED seis) logged oms SNG V p w cdm 
echo -e YAnsiRaD Current Carte SNO " » dare 
echo -e "\n${RED}Machine stats :$NC " ; uptime 
echo -e "\n${RED}Memory stats :$NC " p free 
my 3j 2»&— p 
echo -e "\nS{RED}Local IP Address :SNC" ; echo S(MY IP:-"Not connected") 
echo -e "\n${RED}ISP Address :$NC" ; echo $(MY ISP:-"Not connected") 
echo -e "\n${RED}Open connections :$NC "; netstat -pan inet; 
echo 
} 
# 
# Misc utilities: 


# 


function repeat () 
{ 
local i max 
masc Sl miii 
for ((i-1; i <= 
eval "$Q"; 
done 


function ask() 

( 

eer 
Teame ua 

sexies EES TUR 

*ycpebturm A. 


echo 
case 


-n Vy 


esac 


function corename() 
{ 
ione iibi B. elo 
echo -n Sfill 
done 


max ; 


fm ' 


i Bp 


vr 


# Get name of app that created a corefile. 


S Ge cd 


, 


Repeat n times command. 


HELL 


Se 


, 


gdb 


=p) p. ele 


e "Nn" 


read ans 


# --» C-like syntax 


LOL of us 


xampl 


cor 


Saral lgaicela | 


head 


S 


\ 


PROGRAMMABLE COMPL 


'Bash completion' 


Se ce che ch H HE 


package 


ETITON = ONIE SINCE 
Most are taken from the bash 2.05 documentation and from Ian McDonald's 
(http://www.caliban.org/bash/#completion). 
You will in fact need bash more recent than 3.0 for some features. 


BASH-2.04 


se [ 


"S(BASH VERSIONS. 


epu 


A 


"3." 


l; 


then 


echo "You will need to upgrade to version 3.0 \ 
completion features." 


for full programmabl 
ISIE (brem 


5] 


ny ny ny Su Cal Gn) Cah OI 
(cer SJ) (ox Gn es Go IS) [= 


9 


520 
521 
522 
5/219 
524 
525 
526 
5277] 
528 
529 
530 
Ss 
522 
533 
534 
535 
536 
537 
538 
539 
540 
541 
542 
543 
544 
545 
546 
547 
548 
5/41) 
550) 
55A 
552 
553 
554 
555 
556 
55:7) 
558 
559 
560 
561 
562 
563 
564 
565 
566 
567 
568 
569 
570 
531 
5:072 
573 
574 
575 
576 # Needs the 


fi 


shopt -s ext 
#set +o noun 


complete -A 
complete -A 
complete -A 
complete -A 
complete -A 
complete -A 
complete -A 


complete -A 
complete -A 
complete -A 
complete -A 


complete -A 
complete -A 


# Compressio 


glob 
SEE 


hostname 
export 
variable 
enabled 
alias 
function 
user 


helptopic 
shopt 

stopped -P 
jon = wmv 


directory 
directory 


n 


# Necessary, 


# otherwise some completions will fail. 


iem seio celis loea ie iticjo) joslineg; Cek 


printenv 


export local readonly unset 


Fouls Ae atiav 

alias unalias 
function 

su mail finger 


help # Currently, same as builtins. 


shopt 
Wa bg 


fg jobs disown 


mkdir rmdir 
-o default cd 


zip 

unzip 

compress 

uncompress 

gzip 

gunzip 

bzip2 

bunzip2 
z|Z|gz|GZ|bz2|BZ2)' extract 


gs ghostview ps2pdf ps2ascii 


acroread pdf2ps 


9189222 [| BAZ 573) 3 V Sfr GIONE 


X '!*,texi*' makeinfo texi2dvi texi2html texi2pdf 
X '!*.tex' tex latex slitex 


complet t xo Cleat. =x V sage Zug) V 
complet t go Gerai =x "V. (Zijo|| Aug 
complet E ceo dedgmbpeex etm 
complet Doo Serscbt Ax cL Eva ct 
complet E ie) elenceubllic xe Yer sete EAS 
complet i —© geile =< "wastes e) " 
complet it 9) GelssEEwbE X Ye yar (09272 [1522 | 
complet L CosdefsulE =x tei 2 EZ) 
complet t o Gee =x Yl uale zung | 
i? IDXexeiuWmescNEI = IEXOXSHE C CUciL[9lE e Slo HE p Chala. o oc 
complet p =o Cleiceiwilic =x "UJ oar (oss) " 
complet t -o default -X '!*.+(dvi|DVI) 
complet it c9) Gleegdbe =x "l4 (exe [Jie 
complet t =o Cleirawilic =X \ 

"l* .@(@(? (e) ps|?(E)PS|pdf|PDF) ?(.gz|.GZ| 
complet f -o default 

complet P =© elem 

complet t ox Clemens — Pile oily! Joys 
complet f -o default 

complet E eg) default =x \ 

NSS NEGE 


# Multimedia 


complet IE 


'!*.+(gif|GIF |jp*g |JP*G |bmp |BMP | xpm| XPM 


o default 


XAN 


complet f 


o default 


xc Use ors: || iier 


complet ip 


o default 


X V 9: 4E OCG OGE 


complet His 


'!*.(à (mp[23] |MP[23] logg|OGG|wav|WAV | pls 


o default 


AC ON 


complet f 
' !*,Q0 (mp? (e) 


ps|pes|fli|viv|rm|ram|yuv|mov|MOV| qt | QT 


o default 


XIN 


X a at RN yao LAS 


OE || DOC ||SAle Dab have || WI" | sex || SX? || exssz || CS || exe 2 ODR Ote || Qacar)) "- SOrELSES 


png|PNG)' xv gimp ee gqview 
7 OGLAS SA 
ogg123 


m3u|xm|mod|s[3t]m|it|mtm|ult|flac)' 


g|MP? (E) G| wmalavi|AVI|asf|vob|VOB|bin|dat|voed|^ 


ogm|OGM|mp4|MP4|wav|WAV|asx|ASX)' xine 


complet Hs 


ap MOSES) Sup e 
# a so-calle 


o default 


'universal 
d 'long op 
/—9" gorto 


wmv | mp3 | MP3 | ogg | OGG | \ 


Ap LAN pPSr L perlis 
' completion function - it works when commands have 
jealous; MOCE p- DIES @uliky ahaa! (xs. "ls es 


n of grep 


dvips dvipdf xdvi dviselect dvitype 


xmms 


Sv 
578 


579) a Waist, 


580 
581 
582 
583 
584 
585 
586 
587 
588 
509 
990 
591 
5/92 
593 
594 
595 
596 
59 
598 
599 
600 
601 
602 
603 
604 
605 
606 
607 
608 
609 
610 


ae — (Ex qm 


commented-out version if not available). 


remove '-' from completion word separators 


to work correctly). 


cfd) oly? 


[sor =t 


# (this will allow completions like 'ls color-auto' 
COMP_WORDBREAKS=S { COMP_WORDBREAKS/=/ } 
_get_longopts () 
{ 

#$1 help | sed faa Vel" Is eT sees | 
eeo ^" legs =u p 

Si —-Jwedie | grep -0 —e "—-JI^lsspecesls,]*9" | emus =S "S2 
} 
_longopts () 
{ 

IkeyeeiIk Cts 

cur=S {COMP_WORDS [COMP_CWORD] } 

Case WS (emiegae i}! alin 

=*) oe 
ZU 
25) eee Dese A 
esac 
case "$1" in 
Nes eval emc S EN 
A) Cmag-" SI" pg 

esac 

COMPREPLY-( $( get longopts ${1} $(cur) ) ) 
} 
complete -o default -F _longopts configure bash 
complete -o default -F  longopts wget id info a2ps ls recode 
eae (D) 


{ 


local cur ext regex tar untar 


COMPREPLY= () 
cur=S {COMP_WORDS [COMP_CWORD] } 


# If we want an option, 


return the possible long options. 


case "Scur" in 

=*) COMPREPLY=( $(_get_longopts $1 Scur ) ); return 0;; 
esac 
if | SCOMP_CWORD -eq 1 ]; then 


COMPREPLY-( MiG mule e aso INN 


return 0 


$ ( 


compgen -W 
ia 


case "S(COMP WORDS[1])" in 
? (-)c*£) 
COMPREPLY-( $( compgen -f $cur ) ) 


return 0 


ae (C [L^ 3p) 38), 
ext-'tar' 
regex=Sext 
AEE) 
ext-'tar.gz' 
regex-'tNV(arN.N) N(gzNIZXN) ' 
53s] 918) 


ext-'t?(ar.)bz?(2)' 


-— Seur ) ) 


\ 


643 regex-'tN(arN.N)bz2N?' 


644 r4 

645 ky 

646 COMPREPLY-( $( compgen -f Scur ) ) 

647 return 0 

648 Pi 

649 

650 esac 

651 

652 ane [i |, SCONE ENG) > taret a Uv 5 elem 
653 # Complete on files in tar file. 

654 # 

655 # Get name of tar file from command line. 
656 tar=$( echo "SCOMP_LINE" | \ 

657 eel e Tso WwQ[^ I eese X oS] Nb S 
658 # Devise how to untar and list it. 

659) untar-t$(COMP WORDS[1]//[^Izjyf]/) 

660 

661 COMPREPLY-( $( compgen -W "$( echo $( tar Suntar Star \ 
662 2/(Cev/oell jy jw" == WSeus" y j 
663 return 0 

664 

665 else 

666 # File completion on relevant files. 

667 COMPREPLY-( $( compgen -G $curN*.$ext ) ) 
668 

669 ira 

670 

(5 11. return 0 

CLZ 

Gls } 

674 

675 complete -F tar -o default tar 

676 

677  make() 

678 ( 

679 local mdef makef makef dir-"." makef inc gcmd cur prev i; 
680 COMPREPLY-(); 

681 cur-$(COMP. WORDS [COMP. CWORD] ) ; 

682 prev=S {COMP_WORDS [COMP_CWORD-1] }; 

683 case "Sprev" in 

684 E) 

685 COMPREPLY-($(compgen -f $cur )); 

686 return 0 

687 pay 

688 esac; 

689 Case WsremueW iim 

690 =*) 

691 COMPREPLY-($( get longopts $1 $cur )); 
692 return 0 

693 Pi 

694 esac; 

695 

696 # make reads 'GNUmakefile', then 'makefile', then "Makefile' 
697 if [ -f $(makef dir)/GNUmakefile ]; then 

698 makef=$ {makef_dir}/GNUmakefile 

GOS) elif [ -f ${makef_dir}/makefile ]; then 

700 makef-$(makef dir)/makefile 

701 elif [ -f ${makef_dir}/Makefile ]; then 

702 makef-$(makef dir)/Makefile 

103 else 

704 makef=$ {makef_dir}/*.mk # Local convention. 
705 Tal 

706 

TOT 


708 # Before we scan for targets, see if a Makefile name was 


(€) 


exp cp mp ox] cp md] mdp m cp ml 
(o9; cp von, Onl de (03) NS) [= (9) We) 


# specified with -f 


row (( 


(1=0; uc STESOMESWORPDSTI(GINS iE) do 


if [[ $(COMP WORDS[i]) == -f ]]; then 


# eval for tilde expansion 
eval makef-$(COMP WORDS[i-c1]) 
break 


ira 


done 


[ ! -f $makef ] && return 0 


# deal with included Makefiles 


makef inc-$( grep -E '^-?include' Smakef | \ 
sel =e Ug ^. [Sd hs y 
ror file aim Smaker mep Clo 


[ 


done 


# If we have a partial word to complete, 


-f $file ] && makef="$makef $file" 


# matches of that word. 
ise D x». WSeune! Jo tren ejicinc= Gasp W^Gewuc"U" p elsa gnus 


COMPR 


, 


restrict completions to 


ita 


EPLY-( $( awk -E':' '/^[a-zA-Z0-9][^$4N/NES] *: ([^9] 1$) /. N 
taolit (Sil, 2v, piens (ab sb. A)jeveatiaie JA] PY N 


$makef 2>/dev/null | eval $gcmd 


F mak X '4+($*|*.[cho])' make gmake pmake 


_ heat dl 3E dE (() 


{ 


local 
COMPR 
cur=$ 


# get 


cur prev 
EPLY=() 
{ COMP_WORDS [COMP_CWORD] } 


a list of processes (the first sed evaluation 


# takes care of swapped out processes, the second 
# takes care of getting the basename of the process) 


COMPR 


EPLY-( $( /usr/bin/ps -u SUSER -o comm | \ 


sed -e '1,1d' -e 's#[]\[]##g' -e "s#*.*/##'| \ 
euge "da qe = (A! Seme/)) oeae QI" )») 


return 0 


Gomolerte =T test ililedlial kitien fest iL iljeys 


# A meta-command completion function for commands like sudo(8), 
# first complete on a command, then complete according to that command's own 
# completion definition - currently not quite foolproof, 

# but still quite useful (By Ian McDonald, modified by me). 


_meta_comp () 


local 


COMPR 
us 


cur func cline cspec 
EPLY=() 
{ COMP_WORDS [COMP_CWORD] } 


cmdline=$ {COMP_WORDS [@] } 


)) 


which need to 


IE ie || SCOME TONO = d. Jg ilum 

776 COMPREPLY-( $( compgen -c Scur ) ) 

3373] else 

THS cmd=$ {COMP_WORDS [1] } # Find command. 

779 cspec-$( complete -p $(cmd) ) # Find spec of that command. 
780 

Wea COMP_CWORD and COMP_WORDS() are not read-only, 

782 so we can set them before handing off to regular 
TES completion routine: 

784 Get current command line minus initial command, 
785 eilines" y {COME IIINE S UM 

786 split current command line tokens into array, 

787 COMP WORDS-( Scline ) 

788 set current token number to 1 less than now. 

789 COMP CWORD-$(( $COMP CWORD - 1 )) 

790 Lf Current arg is empty, add ic to COMP WORDS array 
TOL (otherwise that information will be lost). 

7/92 if | -z $cur ]; then COMP. WORDS[COMP CWORD]-2""  ; fi 
793 

794 ie | "Siespescsoo-s" p" Ie VSiesmpec]" Ig iem 

195 # if -F then get function: 

796 func=${cspec#*-F } 

3,89) func=S{func%s%S *} 

798 eval $func $cline # Evaluate it. 

198 else 

800 func-$( echo $cspec | sed -e 's/^complete//' -e 's/[^ ]*$//' ) 
801 COMPREPLY-( $( eval compgen $func $cur ) ) 

802 fua 

803 

804 i3 

805 

806 } 

807 

808 


809 complet o default -F _meta_comp nohup \ 
810 eval exec trace truss strace sotruss gdb 


811 complet o default -F meta comp command type which man nice time 

812 

813 # Local Variables: 

814 # mode:shell-script 

815 # sh-shell:bash 

SLS Gp evel 
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Appendix M. Converting DOS Batch Files to Shell 
Scripts 


Quite a number of programmers learned scripting on a PC running DOS. Even the crippled DOS batch file 
language allowed writing some fairly powerful scripts and applications, though they often required extensive 
kludges and workarounds. Occasionally, the need still arises to convert an old DOS batch file to a UNIX shell 
script. This is generally not difficult, as DOS batch file operators are only a limited subset of the equivalent 
shell scripting ones. 


Table M-1. Batch file keywords / variables / operators, and their shell equivalents 


Meaning 


command-line = DAE prefix 


environmental variable 


comment 


— [3E 


negate following test 


7 


dev/null "black hole" for burying command output 
echo echo (many more option in Bash) 


echo echo blank line 


set +v do not echo command(s) following 


DO [for var in [list]; do "for" loop 
label 


ump to another location in the script 


sleep pause or wait an interval 


menu choice 
if-test 


C 5/5 
E -HE 
© o 10 
| Sle le 
e. o [5 
[e] £5 S 2 
e 2s 
BS |= 
2. |S 
O JS 
5 
— 


IST FILENAME 


= 
= 

a 
€ 
zh 
— 
ie 
5 
5 
© 


] test if file exists 


if [ -z "$N" ] if replaceable parameter "N" not present 


source or . (dot operator) "include" another script 


COMMAND /C source or . (dot operator) "include" another script (same as CALL) 


export set an environmental variable 


shift left shift command-line argument list 


SGN -lt or -gt sign (of integer) 


ERRORLEVEL $? exit status 
stdin "console" (st din) 


(generic) printer device 


first printer device 


Batch files usually contain DOS commands. These must be translated into their UNIX equivalents in order to 
convert a batch file into a shell script. 


Table M-2. DOS commands and their UNIX equivalents 


DOS Command |UNIX Equivalent 


OQ 
p] [ct 
E 


[s] 
BN 
E! 

E 
"Jj 
LT] 

L1] 


$^; Virtually all UNIX and shell operators and commands have many more options and enhancements than 
their DOS and batch file counterparts. Many DOS batch files rely on auxiliary utilities, such as ask.com, 


a crippled counterpart to read. 


DOS supports only a very limited and incompatible subset of filename wild-card expansion, recognizing 
just the * and ? characters. 
Converting a DOS batch file into a shell script is generally straightforward, and the result ofttimes reads better 
than the original. 


Example M-1. VIEWDATA.BAT: DOS Batch File 


REM VIEWDATA 


EM INSPIRED BY AN EXAMPLE IN "DOS POWERTOOLS" 


P. 
P. 


EM BY PAUL SOMERSON 


(O) 


ECHORORH 


F !$1--! GOTO VIEWDATA 

E IF NO COMMAND-LINE ARG... 
IND "$1" C:\BOZO\BOOKLIST.TXT 
OTO EXITO 
E PRINT LINE WITH STRING MATCH, THEN EXIT. 


EWDATA 
E C:\BOZO\BOOKLIST.TXT | MORI 
REM SHOW ENTIRE FILE, 1 PAGE AT A TIME. 


E 


|ES 


ey Cs} | xy (a) dex Ge) IS) f=) €» Wey Csi cp feny (Oan dE (5 ISS) [3 


:EXITO 


The script conversion is somewhat of an improvement. [1] 


Example M-2. viewdata.sh: Shell Script Conversion of VIEWDATA.BAT 


#!/bin/bash 
# viewdata.sh 
# Conversion of VIEWDATA.BAT to shell script. 


DATAFILE=/home/bozo/datafiles/book-collection.data 
ARGNO-1 


# QECHO OFF Command unnecessary here. 


1 
2 
3 
4 
5 
6 
7 
8 
S 
10 if [ $4 —-lt "SARGNO" ] IF !$1--2! GOTO VIEWDATA 
11 then 
2 
3 
4 
5 
6 
p 
8 
9 
0 
il 


less SDATAFILI 
else 

grep "$1" SDATAFILI 
ial 


RI 


TYPI 


Dm 


C:\MYDIR\BOOKLIST.TXT | MORE 


TIND WILY (Cog MOI) IR \ EVOKE LSI, WOT 


E 


esac O EX TRO 


# GOTOs, labels, smoke-and-mirrors, and flimflam unnecessary. 
# The converted script is short, sweet, and clean, 
#+ which is more than can be said for the original. 


Ted Davis' Shell Scripts on the PC site has a set of comprehensive tutorials on the old-fashioned art of batch 
file programming. Certain of his ingenious techniques could conceivably have relevance for shell scripts. 


Notes 


[1] Various readers have suggested modifications of the above batch file to prettify it and make it more 
compact and efficient. In the opinion of the ABS Guide author, this is wasted effort. A Bash script can 
access a DOS filesystem, or even an NTFS partition (with the help of ntfs-3g) to do batch or scripted 


operations. 
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Appendix N. Exercises 


The exercises that follow test and extend your knowledge of scripting. Think of them as a challenge, as an 
entertaining way to take you further along the stony path toward UNIX wizardry. 


On a dingy side street in a run-down section of Hoboken, New Jersey, 
there sits a nondescript squat two-story brick building with an inscription 
incised on a marble plate in its wall: 


Bash Scripting Hall of Fame. 


Inside, among various dusty uninteresting exhibits is a corroding, 
cobweb-festooned brass plaque inscribed with a short, very short 

list of those few persons who have successfully mastered the material 

in the Advanced Bash Scripting Guide, as evidenced by their performance 
on the following Exercise sections. 


(Alas, the author of the ABS Guide is not represented among the exhibits. 
This is possibly due to malicious rumors about lack of credentials and 


deficient scripting skills.) 


N.1. Analyzing Scripts 


Examine the following script. Run it, then explain what it does. Annotate the script and rewrite it in a more 
compact and elegant manner. 


1 #!/bin/bash 

2 

3 MAX-10000 

4 

E 

6 for((nr-l; nr«$MAX; nr+t) ) 
F do 

8 

E lee “ed = nre z 5" 
10 xit [ "Sel? -ne 3 ] 
il then 

2 continue 

3 al 

4 

E dct Eme s n 
6 we qp USZU mme 4] 
Ji then 

8 continue 

i9 3E ab 
20 
21 Tet TES = Nre %7 97 
22 sae [ "SES" me 5] 
23 then 
24 continue 
25 fa 
26 


27 break # What happens when you comment out this line? Why? 


29 done 

30 

il echo "Number = Snr" 
32 

33 

34 exit d) 


Explain what the following script does. It is really just a parameterized command-line pipe. 


#!/bin/bash 


DIRNAME=/usr/bin 
FILETYPE="Shell script" 
LOGFILE=logfile 


file "SDIRNAME"/* | fgrep "SFILETYPE" | tee $LOGFILE | wc -1l 


oy (99^ s] fn (Gal dE (9 [Soy [S 


exe 0) 


Examine and explain the following script. For hints, you might refer to the listings for find and stat. 


#!/bin/bash 


# This code is released to the public domain. 


1L 

2 

3 # Author: Nathan Coulter 

4 

5 # The author gave permission to use this code snippet in the ABS Guide. 
6 


7 filme! —weeswolepicim i1 -type E -princi "SWOOU"' || 4 
8 while read -d $'\000'; do 


9 mv "SREPLY" "$(date -d "$(stat © '%y' "SREPLY") " '+%Y%m%d3H%M%S' 
10 )-SREPLY" 
jT. done 
12 } 
13} 


14 # Warning: Test-drive this script in a "scratch" directory. 
15 # It will somehow affect all the files there. 


A reader sent in the following code snippet. 


1 while read LINE 
Z. elo 
3 echo $LINE 
4 done « ^tail -f /var/log/messages' 
He wished to write a script tracking changes to the system log file, /var/log/messages. Unfortunately, 
the above code block hangs and does nothing useful. Why? Fix this so it does work. (Hint: rather than 


redirecting the stdin of the loop, try a pipe.) 


Analyze the following "one-liner" (here split into two lines for clarity) contributed by Rory Winston: 


il ewgoowiE SUNO; for © ium Sime see -—mewue Uc sese) 
2 elo exo SUM=S ( (SSUM r Se -L Si || ws VI prime Sil QU) themes echo SSUM 


Hint: First, break the script up into bite-sized sections. Then, carefully examine its use of double-parentheses 
arithmetic, the export command, the find command, the wc command, and awk. 


Analyze Example A-10, and reorganize it in a simplified and more logical style. See how many of the 
variables can be eliminated, and try to optimize the script to speed up its execution time. 


Alter the script so that it accepts any ordinary ASCII text file as input for its initial "generation". The script 
will read the first SROW*SCOL characters, and set the occurrences of vowels as "living" cells. Hint: be sure to 
translate the spaces in the input file to underscore characters. 
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N.2. Writing Scripts 


Write a script to carry out each of the following tasks. 
EASY 


Self-reproducing Script 
Write a script that backs itself up, that is, copies itself to a file named backup. sh. 


Hint: Use the cat command and the appropriate positional parameter. 

Home Directory Listing 
Perform a recursive directory listing on the user's home directory and save the information to a file. 
Compress the file, have the script prompt the user to insert a USB flash drive, then press ENTER. 
Finally, save the file to the flash drive after making certain the flash drive has properly mounted by 
parsing the output of df. 

Converting for loops to while and until loops 
Convert the for loops in Example 11-1 to while loops. Hint: store the data in an array and step through 
the array elements. 


Having already done the "heavy lifting," now convert the loops in the example to until loops. 
Changing the line spacing of a text file 

Write a script that reads each line of a target file, then writes the line back to stdout, but with an 

extra blank line following. This has the effect of double-spacing the file. 


Include all necessary code to check whether the script gets the necessary command-line argument (a 
filename), and whether the specified file exists. 


When the script runs correctly, modify it to triple-space the target file. 


Finally, write a script to remove all blank lines from the target file, single-spacing it. 

Backwards Listing 
Write a script that echoes itself to st dout, but backwards. 

Automatically Decompressing Files 
Given a list of filenames as input, this script queries each target file (parsing the output of the file 
command) for the type of compression used on it. Then the script automatically invokes the 
appropriate decompression command (gunzip, bunzip2, unzip, uncompress, or whatever). If a target 
file is not compressed, the script emits a warning message, but takes no other action on that particular 
file. 

Unique System ID 
Generate a "unique" 6-digit hexadecimal identifier for your computer. Do not use the flawed hostid 
command. Hint: md5sum /etc/passwd, then select the first 6 digits of output. 

Backup 
Archive as a "tarball" (* .tar.gz file) all the files in your home directory tree 
(/home/your-name) that have been modified in the last 24 hours. Hint: use find. 

Checking whether a process is still running 
Given a process ID (PID) as an argument, this script will check, at user-specified intervals, whether 
the given process is still running. You may use the ps and sleep commands. 

Primes 
Print (to stdout) all prime numbers between 60000 and 63000. The output should be nicely 
formatted in columns (hint: use printf). 

Lottery Numbers 


One type of lottery involves picking five different numbers, in the range of 1 - 50. Write a script that 
generates five pseudorandom numbers in this range, with no duplicates. The script will give the 
option of echoing the numbers to st dout or saving them to a file, along with the date and time the 
particular number set was generated. (If your script consistently generates winning lottery numbers, 
then you can retire on the proceeds and leave shell scripting to those of us who have to work for a 
living.) 


INTERMEDIATE 


Integer or String 
Write a script function that determines if an argument passed to it is an integer or a string. The 
function will return TRUE (0) if passed an integer, and FALSE (1) if passed a string. 


Hint: What does the following expression return when $1 is not an integer? 


expr $1 + 0 

Managing Disk Space 
List, one at a time, all files larger than 100K in the /home/username directory tree. Give the user 
the option to delete or compress the file, then proceed to show the next one. Write to a logfile the 
names of all deleted files and the deletion times. 

Banner 
Simulate the functionality of the deprecated banner command in a script. 

Removing Inactive Accounts 
Inactive accounts on a network waste disk space and may become a security risk. Write an 
administrative script (to be invoked by root or the cron daemon) that checks for and deletes user 
accounts that have not been accessed within the last 90 days. 

Enforcing Disk Quotas 
Write a script for a multi-user system that checks users' disk usage. If a user surpasses a preset limit 
(100 MB, for example) in her /home/username directory, then the script automatically sends her a 
warning e-mail. 


The script will use the du and mail commands. As an option, it will allow setting and enforcing quotas 
using the quota and setquota commands. 

Logged in User Information 
For all logged in users, show their real names and the time and date of their last login. 


Hint: use who, lastlog, and parse /etc/passwd. 

Safe Delete 
Implement, as a script, a "safe" delete command, sde1. sh. Filenames passed as command-line 
arguments to this script are not deleted, but instead gzipped if not already compressed (use file to 
check), then moved to a ~/ TRASH directory. Upon invocation, the script checks the ~/ TRASH 
directory for files older than 48 hours and permanently deletes them. (An better alternative might be 
to have a second script handle this, periodically invoked by the cron daemon.) 


Extra credit: Write the script so it can handle files and directories recursively. This would give it the 
capability of "safely deleting" entire directory structures. 

Making Change 
What is the most efficient way to make change for $1.68, using only coins in common circulations 
(up to 25c)? It's 6 quarters, 1 dime, a nickel, and three cents. 


Given any arbitrary command-line input in dollars and cents ($*.??), calculate the change, using the 
minimum number of coins. If your home country is not the United States, you may use your local 
currency units instead. The script will need to parse the command-line input, then change it to 
multiples of the smallest monetary unit (cents or whatever). Hint: look at Example 24-8. 


Quadratic Equations 
Solve a quadratic equation of the form Ax^2 + Bx + C = 0. Have a script take as arguments the 
coefficients, A, B, and C, and return the solutions to four decimal places. 


Hint: pipe the coefficients to bc, using the well-known formula, x = ( -B 4/- sqrt( B^2 - 
4AC ) ) / 2A. 

Table of Logarithms 
Using the bc and printf commands, print out a nicely-formatted table of eight-place natural logarithms 
in the interval between 0.00 and 100.00, in steps of .01. 


Hint: bc requires the —1 option to load the math library. 

Sum of Matching Numbers 
Find the sum of all five-digit numbers (in the range 10000 - 99999) containing exactly two out of the 
following set of digits: ( 4, 5, 6 }. These may repeat within the same number, and if so, they count 
once for each occurrence. 


Some examples of matching numbers are 42057, 74638, and 89515. 

Lucky Numbers 
A lucky number is one whose individual digits add up to 7, in successive additions. For example, 
62431 is a lucky number (6 +2 - 4-3 1216,14 6- 7). Find all the lucky numbers between 1000 
and 10000. 

Craps 
Borrowing the ASCII graphics from Example A-40, write a script that plays the well-known gambling 
game of craps. The script will accept bets from one or more players, roll the dice, and keep track of 
wins and losses, as well as of each player's bankroll. 

Tic-tac-toe 
Write a script that plays the child's game of tic-tac-toe against a human player. The script will let the 
human choose whether to take the first move. The script will follow an optimal strategy, and therefore 
never lose. To simplify matters, you may use ASCII graphics: 


i o | = | 

= n m 

3 lox | 

7 ut 

5 | 9 | 

6 

d Your move, human (row, column)? 
Alphabetizing a String 


Alphabetize (in ASCII order) an arbitrary string read from the command-line. 

Parsing 
Parse /etc/passwd, and output its contents in nice, easy-to-read tabular form. 

Logging Logins 
Parse /var/log/messages to produce a nicely formatted file of user logins and login times. The 
script may need to run as root. (Hint: Search for the string "LOGIN.") 

Pretty-Printing a Data File 
Certain database and spreadsheet packages use save-files with comma-separated values (CSVs). 
Other applications often need to parse these files. 


Given a data file with comma-separated fields, of the form: 


1 Jones,Bill,235 S. Williams St.,Denver,CO,80221,(303) 244-7989 
2 Smith,Tom,404 Polk Ave.,Los Angeles,CA,90003,(213) 879-5612 
DIT 
Reformat the data and print it out to stdout in labeled, evenly-spaced columns. 
Justification 


Given ASCII text input either from st din or a file, adjust the word spacing to right-justify each line 
to a user-specified line-width, then send the output to stdout. 

Mailing List 
Using the mail command, write a script that manages a simple mailing list. The script automatically 
e-mails the monthly company newsletter, read from a specified text file, and sends it to all the 
addresses on the mailing list, which the script reads from another specified file. 

Generating Passwords 
Generate pseudorandom 8-character passwords, using characters in the ranges [0-9], [A-Z], [a-z]. 
Each password must contain at least two digits. 

Monitoring a User 
You suspect that one particular user on the network has been abusing his privileges and possibly 
attempting to hack the system. Write a script to automatically monitor and log his activities when he's 
signed on. The log file will save entries for the previous week, and delete those entries more than 
seven days old. 


You may use last, lastlog, and lastcomm to aid your surveillance of the suspected malefactor. 
Checking for Broken Links 
Using lynx with the -traversal option, write a script that checks a Web site for broken links. 


DIFFICULT 


Testing Passwords 
Write a script to check and validate passwords. The object is to flag "weak" or easily guessed 
password candidates. 


A trial password will be input to the script as a command-line parameter. To be considered 
acceptable, a password must meet the following minimum qualifications: 


© Minimum length of 8 characters 

9 Must contain at least one numeric character 

© Must contain at least one of the following non-alphabetic characters: @, #, $, 96, &, *, +, -, = 
Optional: 


0 Do a dictionary check on every sequence of at least four consecutive alphabetic characters in 
the password under test. This will eliminate passwords containing embedded "words" found 
in a standard dictionary. 

9 Enable the script to check all the passwords on your system. These probably do not reside in 
/etc/passwd. 

This exercise tests mastery of Regular Expressions. 
Cross Reference 
Write a script that generates a cross-reference (concordance) on a target file. The output will be a 
listing of all word occurrences in the target file, along with the line numbers in which each word 
occurs. Traditionally, linked list constructs would be used in such applications. Therefore, you should 
investigate arrays in the course of this exercise. Example 16-12 is probably not a good place to start. 
Square Root 
Write a script to calculate square roots of numbers using Newton's Method. 


The algorithm for this, expressed as a snippet of Bash pseudo-code is: 


# (Isaac) Newton's Method for speedy extraction 
#+ of square roots. 


guess = Sargument 
# Sargument is the number to find the square root of. 
t Sguess is each successive calculated "guess" -- or trial solution -- 


ep (Gal dex (E39) fs [5 


+ of the square root. 
Our first "guess" at a square root is the argument itself. 


oldguess = 0 
Soldguess is the previous $guess. 


tolerance - .000001 
To how close a tolerance we wish to calculate. 


loopcnt = 0 


PRPPPPP PPE 
O00 -1 OY Ui 4A €) M PB O «o 0 -1 


Let's keep track of how many times through the loop. 

Some arguments will require more loop iterations than others. 
19 
20 
21 while [ ABS( $guess $oldguess ) -gt Stolerance ] 
22 # ORS SES RES OOS R Up Syn ean O C OLE SCs 
23 
24 # "ABS" is a (floating point) function to find the absolute value 
25 #+ of the difference between the two terms. 
26 # So, as long as difference between current and previous 
27 #+ trial solution (guess) exceeds the tolerance, keep looping. 
28 

do 
oldguess = $guess # Update Soldguess to previous $guess. 
guess = | Soldguess + | $argument / $oldguess ) ) / 2.0 
= 1/2 ( (Soldguess **2 + Sargument) / Soldguess ) 


equivalent to: 
= 1/2 ( $oldguess + S$argument / Soldguess ) 
that is, "averaging out" the trial solution and 
* the proportion of argument deviation 
+ (in effect, splitting the error in half). 


CO CO CO CO CO CO CO CO CO CO Dd 
(oy (oer c (exp (Gal (ES (Go) ISS) =) em). (o) 


40 This converges on an accurate solution 

41 #+ with surprisingly few loop iterations 

42 #+ for arguments > S$tolerance, of course. 

43 

44 

AS (( looses }) )) # Update loop counter. 
46 done 


It's a simple enough recipe, and seems at first glance easy enough to convert into a working Bash 
script. The problem, though, is that Bash has no native support for floating point numbers. So, the 
script writer needs to use bc or possibly awk to convert the numbers and do the calculations. It may 
get rather messy... 

Logging File Accesses 
Log all accesses to the files in /et c during the course of a single day. This information should 
include the filename, user name, and access time. If any alterations to the files take place, that should 
be flagged. Write this data as neatly formatted records in a logfile. 

Monitoring Processes 
Write a script to continually monitor all running processes and to keep track of how many child 
processes each parent spawns. If a process spawns more than five children, then the script sends an 
e-mail to the system administrator (or root) with all relevant information, including the time, PID of 
the parent, PIDs of the children, etc. The script appends a report to a log file every ten minutes. 

Strip Comments 
Strip all comments from a shell script whose name is specified on the command-line. Note that the #! 
line must not be stripped out. 

Strip HTML Tags 
Strip all HTML tags from a specified HTML file, then reformat it into lines between 60 and 75 
characters in length. Reset paragraph and block spacing, as appropriate, and convert HTML tables to 
their approximate text equivalent. 

XML Conversion 


Convert an XML file to both HTML and text format. 

Chasing Spammers 
Write a script that analyzes a spam e-mail by doing DNS lookups on the IP addresses in the headers to 
identify the relay hosts as well as the originating ISP. The script will forward the unaltered spam 
message to the responsible ISPs. Of course, it will be necessary to filter out your own ISP's IP 
address, so you don't end up complaining about yourself. 


As necessary, use the appropriate network analysis commands. 
For some ideas, see Example 16-41 and Example A-28. 


Optional: Write a script that searches through a list of e-mail messages and deletes the spam 
according to specified filters. 

Creating man pages 
Write a script that automates the process of creating man pages. 


Given a text file which contains information to be formatted into a man page, the script will read the 
file, then invoke the appropriate groff commands to output the corresponding man page to stdout. 
The text file contains blocks of information under the standard man page headings, i.e., NAME, 
SYNOPSIS, DESCRIPTION, etc. 


Example A-39 is an instructive first step. 

Morse Code 
Convert a text file to Morse code. Each character of the text file will be represented as a 
corresponding Morse code group of dots and dashes (underscores), separated by whitespace from the 
next. For example: 


1 Invoke the "morse.sh" script with "script" 
2 as an argument to convert to Morse. 


3 

4 

5 $ sh morse.sh script 

6 

7 

ONES [s ie a p iE 
Hex Dump 


Do a hex(adecimal) dump on a binary file specified as an argument. The output should be in neat 
tabular fields, with the first field showing the address, each of the next 8 fields a 4-byte hex number, 
and the final field the ASCII equivalent of the previous 8 fields. 


The obvious followup to this is to extend the hex dump script into a disassembler. Using a lookup 
table, or some other clever gimmick, convert the hex values into 80x86 op codes. 

Emulating a Shift Register 
Using Example 27-15 as an inspiration, write a script that emulates a 64-bit shift register as an array. 
Implement functions to load the register, shift left, shift right, and rotate it. Finally, write a function 
that interprets the register contents as eight 8-bit ASCII characters. 

Calculating Determinants 
Write a script that calculates determinants [1] by recursively expanding the minors. Use a4 x 4 
determinant as a test case. 

Hidden Words 
Write a "word-find" puzzle generator, a script that hides 10 input words in a 10 x 10 array of random 
letters. The words may be hidden across, down, or diagonally. 


Optional: Write a script that solves word-find puzzles. To keep this from becoming too difficult, the 
solution script will find only horizontal and vertical words. (Hint: Treat each row and column as a 


string, and search for substrings.) 

Anagramming 
Anagram 4-letter input. For example, the anagrams of word are: do or rod row word. You may use 
/usr/share/dict/linux.words as the reference list. 

Word Ladders 
A "word ladder" is a sequence of words, with each successive word in the sequence differing from the 
previous one by a single letter. 


For example, to "ladder" from mark to vase: 


L neucis ==> pakk ==> part > Test > weg -> weise 


2 ^ ^ ^ 


Write a script that solves word ladder puzzles. Given a starting and an ending word, the script will list 
all intermediate steps in the "ladder." Note that all words in the sequence must be legitimate 
dictionary words. 

Fog Index 
The "fog index" of a passage of text estimates its reading difficulty, as a number corresponding 
roughly to a school grade level. For example, a passage with a fog index of 12 should be 
comprehensible to anyone with 12 years of schooling. 


The Gunning version of the fog index uses the following algorithm. 


- 


. Choose a section of the text at least 100 words in length. 

Count the number of sentences (a portion of a sentence truncated by the boundary of the text 
section counts as one). 

. Find the average number of words per sentence. 


ma 


W 


AVE_WDS_SEN = TOTAL_WORDS / SENTENCES 
4. Count the number of "difficult" words in the segment -- those containing at least 3 syllables. 
Divide this quantity by total words to get the proportion of difficult words. 


PRO_DIFF_WORDS = LONG_WORDS / TOTAL_WORDS 
. The Gunning fog index is the sum of the above two quantities, multiplied by 0.4, then 
rounded to the nearest integer. 


CA 


G FOG INDEX = int (0.4 * ( AVE WDS SEN + PRO DIFF WORDS ) ) 
Step 4 is by far the most difficult portion of the exercise. There exist various algorithms for estimating 
the syllable count of a word. A rule-of-thumb formula might consider the number of letters in a word 
and the vowel-consonant mix. 


A strict interpretation of the Gunning fog index does not count compound words and proper nouns as 
"difficult" words, but this would enormously complicate the script. 

Calculating PI using Buffon's Needle 
The Eighteenth Century French mathematician de Buffon came up with a novel experiment. 
Repeatedly drop a needle of length n onto a wooden floor composed of long and narrow parallel 
boards. The cracks separating the equal-width floorboards are a fixed distance d apart. Keep track of 
the total drops and the number of times the needle intersects a crack on the floor. The ratio of these 
two quantities turns out to be a fractional multiple of PI. 


In the spirit of Example 16-50, write a script that runs a Monte Carlo simulation of Buffon's Needle. 
To simplify matters, set the needle length equal to the distance between the cracks, n = d. 


Hint: there are actually two critical variables: the distance from the center of the needle to the nearest 
crack, and the inclination angle of the needle to that crack. You may use bc to handle the calculations. 


Playfair Cipher 
Implement the Playfair (Wheatstone) Cipher in a script. 


The Playfair Cipher encrypts text by substitution of digrams (2-letter groupings). It is traditional to 
use a 5 x 5 letter scrambled-alphabet key square for the encryption and decryption. 


C Q D BS 
A E m (E mi 
JE JE dup AE hi 
ig OR a uj 
WW OS XE E 


Each letter of the alphabet appears once, except "I" also represents 
"J". The arbitrarily chosen key word, "CODES" comes first, then all 

the rest of the alphabet, in order from left to right, skipping letters 
already used. 


To encrypt, separate the plaintext message into digrams (2-letter 
groups). If a group has two identical letters, delete the second, and 
[s 


form a new group. If there is a single letter left over at the end, 
Taecke @ ÜhowndLILW. character; iryyjeueeliky am Wc. 


TASIS TAS TOP SECRET “MESSAGE 


Wel IS; 30$. ANIL Ole’ (Sls, (CMe IN MO, SVN (6jn 


NPRPRPPRP PPP PY 
O (o 0» -1 O Oi i$ (). NM PS. O xo 0 -1 O O1 4 CQ Io E 


21 

22 

23 For each digram, there are three possibilities. 

24 

25 

26 1) Both letters will be on the same row of the key square: 

2 For each letter, substitute the one immediately to the right, in that 
28 row. If necessary, wrap around left to the beginning of the row. 
29 

30 or 

Sil 

32 2) Both letters will be in the same column of the key square: 


For each letter, substitute the one immediately below it, in that 
row. If necessary, wrap around to the top of the column. 


3) Both letters will form the corners of a rectangle within the key square: 
For each letter, substitute the one on the other corner the rectangl 


WWW CO CO CO CO 
O 0 IHD Bf Ww 
[9] 
hn 


40 which lies on the same row. 
41. 
42 
43 The "TH" digram falls under case #3. 
44 GH 
45 MN 
Te. Tp (Rectangle with "T" and "H" at corners) 
47 
4s T ==> U 
49 H --> G 
50 
5L 
52 The "SE" digram falls under case #1. 
DATO Os Siege (Row (GOES SY sunei Wa!) 
S => C (wraps around left to beginning of row) 
S ==> f$ 


(On font (yh nh (n (Sm (Oni 
Go} ceo) «| (ex (On ds ey) 


60 To decrypt encrypted text, reverse the above procedure under cases #1 
61 and 42 (move in opposite direction for substitution). Under case 43, 

62 just take the remaining two corners of the rectangle. 

63 
64 
65 Helen Fouche Gaines' classic work, ELEMENTARY CRYPTANALYSIS (1939), gives a 
66 fairly detailed description of the Playfair Cipher and its solution methods. 


This script will have three main sections 


I. Generating the key square, based on a user-input keyword. 
II. Encrypting a plaintext message. 
III. Decrypting encrypted text. 
The script will make extensive use of arrays and functions. 


Please do not send the author your solutions to these exercises. There are more appropriate ways to impress 
him with your cleverness, such as submitting bugfixes and suggestions for improving this book. 


Notes 


[1] For all you fine people who failed second-year algebra, a determinant is a numerical quantity associated 
with a multidimensional matrix (array of numbers). 


1 For the simple case of a 2 x 2 determinant: 

2 

3 la bl 

4 |b al 

5 

6 The solution is a*a - b*b, where "a" and "b" represent numbers. 
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Appendix O. Revision History 


This document first appeared as a 60-page HOWTO in the late spring 
of 2000. Since then, it has gone through quite a number of updates 
and revisions. This book could not have been written without the 
assistance of the Linux community, and especially of the volunteers 
of the Linux Documentation Project. 


Here is the e-mail to the LDP requesting permission to submit version 0.1. 


From thegrendel@theriver.com Sat Jun 10 09:05:33 2000 -0700 
Deter Sat, 10 Jum 2000 0905-25 =O700) (MST) 

From: "M. Leo Cooper" <thegrendel@theriver.com> 

X-Sender: thegrendel@localhost 

To: ldp-discuss@lists.linuxdoc.org 

Subject: Permission to submit HOWTO 


Dear HOWTO Coordinator, 


I am working on and would like to submit to the LDP a HOWTO on the subject 
Or isum S\Kerenjocaing'! (shell seriene; Usine Hasht). AS wie happens, 

I have been writing this document, off and on, for about the last eight 
MOMENS gus SO, Amc i. (poule peel. a cirst Creare du ASCII rart Tommet. 30m 

a matter of just a few more days. 


I began writing this out of frustration at being unable to find a 
decent book on shell scripting. I managed to locate some pretty good 
articles on various aspects of scripting, but nothing like a complete, 
beginning-to-end tutorial. Well, in keeping with my philosophy, if all 
else fails, do it yourself. 


Mop pop opp PPP PY 
O io 00 -1 O Oi i$ (0. M HIS. O xo 0 -1 O O1 4S CQ I ES 


N N 
i) [= 


As it stands, this proposed "Bash-Scripting HOWTO" would serve as a 
combination tutorial and reference, with the heavier emphasis on the 
tutorial. It assumes Linux experience, but only a very basic level 
of programming skills. Interspersed with the text are 79 illustrative 


SOURS: 
(Ont gm w 


26 example scripts of varying complexity, all liberally commented. There 

ZI eue ven exercises for the reader. 

28 

29 At this stage, I'm up to 18,000+ words (124k), and that's over 50 pages of 
30 text (whew!). 

3d 

32 

33 I haven't mentioned that I've previously authored an LDP HOWTO, the 

34 "Software-Building HOWTO", which I wrote in Linuxdoc/SGML. I don't know 
35 if I could handle Docbook/SGML, and I'm glad you have volunteers to do 
36 the conversion. You people seem to have gotten on a more organized basis 
37 these last few months. Working with Greg Hankins and Tim Bynum was nice, 
38 but a professional team is even nicer. 

39 


Anyhow, please advise. 


Mendel Cooper 
thegrendel@theriver.com 


PP bP SP 
Hs GS [m$ [23 G2» 


Table O-1. Revision History 


Release Date Comments 
0.1 14 Jun 2000 Initial release. 
0.2 30 Oct 2000 Bugs fixed, plus much additional material and more example scripts. 


PPP PrP OOO 


12 Feb 2001 
08 Jul 2001 
03 Sep 2001 
14 Oct 2001 
06 Jan 2002 
31 Mar 2002 
02 Jun 2002 
16 Jun 2002 
13 Jul 2002 
29 Sep 2002 
05 Jan 2003 
10 May 2003 


Major update. 

Complete revision and expansion of the book. 

Major update: Bugfixes, material added, sections reorganized. 

Stable release: Bugfixes, reorganization, material added. 

Bugfixes, material and scripts added. 

Bugfixes, material and scripts added. 

TANGERINE release: A few bugfixes, much more material and scripts added. 
MANGO release: A number of typos fixed, more material and scripts. 
PAPAYA release: A few bugfixes, much more material and scripts added. 
POMEGRANATE telease: Bugfixes, more material, one more script. 
COCONUT release: A couple of bugfixes, more material, one more script. 


BREADFRUIT release: A number of bugfixes, more scripts and material. 


a 


a 


a 


BP WWW WW Wo WwW WW Co P2 P2 NY NY NY NY NY NY NY EF ! 
OD Oi! i$ WN FP OWN LP DOO OBA DO BP WN FP OO OO -] ON OA 4 WN FP DMO WANA Os OA 4 WN FP 0o OC BK W 
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21 Jun 2003 
24 Aug 2003 
14 Sep 2003 
31 Oct 2003 
03 Jan 2004 
25 Jan 2004 
15 Feb 2004 
15 Mar 2004 
18 Apr 2004 
11 Jul 2004 
03 Oct 2004 
14 Nov 2004 
06 Feb 2005 
20 Mar 2005 
08 May 2005 
05 Jun 2005 
28 Aug 2005 
23 Oct 2005 
26 Feb 2006 
15 May 2006 
18 Jun 2006 
08 Oct 2006 
10 Dec 2006 
29 Apr 2007 
24 Jun 2007 
10 Nov 2007 
16 Mar 2008 
11 May 2008 
21 Jul 2008 
23 Nov 2008 
26 Jan 2009 
23 Mar 2009 


PERSIMMON release: Bugfixes, and more material. 
GOOSEBERRY release: Major update. 
HUCKLEBERRY release: Bugfixes, and more material. 
CRANBERRY release: Major update. 
STRAWBERRY release: Bugfixes and more material. 
MUSKMELON release: Bugfixes. 

STARFRUIT release: Bugfixes and more material. 
SALAL release: Minor update. 

MULBERRY release: Minor update. 

ELDERBERRY release: Minor update. 
LOGANBERRY release: Major update. 

BAYBERRY release: Bugfix update. 

BLUEBERRY release: Minor update. 

RASPBERRY release: Bugfixes, much material added. 
TEABERRY release: Bugfixes, stylistic revisions. 
BOXBERRY release: Bugfixes, some material added. 
POKEBERRY release: Bugfixes, some material added. 
WHORTLEBERRY release: Bugfixes, some material added. 
BLAEBERRY release: Bugfixes, some material added. 
SPICEBERRY release: Bugfixes, some material added. 
WINTERBERRY release: Major reorganization. 
WAXBERRY release: Minor update. 
SPARKLEBERRY release: Important update. 
INKBERRY release: Bugfixes, material added. 
SERVICEBERRY release: Major update. 
LINGONBERRY release: Minor update. 
SILVERBERRY release: Important update. 
GOLDENBERRY release: Minor update. 
ANGLEBERRY release: Major update. 
FARKLEBERRY release: Minor update. 
WORCESTERBERRY release: Minor update. 
THIMBLEBERRY release: Major update. 


6.1 30 Sep 2009 BUFFALOBERRY release: Minor update. 
6:2 17Mar2010 ROWANBERRY release: Minor update. 
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Appendix P. Download and Mirror Sites 


The latest update of this document, as an archived, bzip2-ed "tarball" including both the SGML source and 
rendered HTML, may be downloaded from the author's home site). A pdf version is also available. The 
change log gives a detailed revision history. The ABS Guide even has its own £reshmeat.net page to 
keep track of major updates, user comments, and popularity ratings for the project. 


The main hosting site for this document is the Linux Documentation Project, which maintains many other 
Guides and HOWTOs as well. 


Many thanks to Ronny Bangsund for donating server space to host this project. 
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Appendix Q. To Do List 


* A comprehensive survey of incompatibilities between Bash and the classic Bourne shell. 
e Same as above, but for the Korn shell (ksh). 
* A primer on CGI programming, using Bash. 


Here is a simple CGI script to get you started. 


Example Q-1. Print the server environment 


1 #!/bin/bash 
2 May have to change the location for your site. 
3 (At the ISP's servers, Bash may not be in the usual place.) 
4 Other places: /usr/bin or /usr/local/bin 
5 Might even try it without any path in sha-bang. 
6 
7 test-cgi.sh 
8 by Michael Zick 
9 Used with permission 

10 

abt 

12 Disable filename globbing. 

ks) See i 

14 

L5 Header tells browser what to expect. 

16 echo Content-type: text/plain 

17 echo 

18 

IY) secho CEr/L.-0 test Seiler ieejooucc § 

20 echo 

21 

22 echo environment settings: 

23 set 

24 echo 

25 

26 echo whereis bash? 

27 whereis bash 

28 echo 

29 

30 

3l echo who are we? 

32 echo $(BASH VERSINFO[*]) 

33 echo 

34 

35 GEO mug is Si. arov ag UU. 

36 echo 

Si 

38 # CGI/1.0 expected environment variables. 

ED 

40 echo SERVER SOFTWARE = S$SERVER SOFTWARE 

41 echo SERVER NAME = $SERVER NAME 

42 echo GATEWAY INTERFACE = $GATEWAY INTERFACE 

43 echo SERVER PROTOCOL = S$SERVER PROTOCOL 

44 echo SERVER PORT = S$SERVER, PORT 

45 echo REQUEST METHOD - SREQUEST METHOD 

46 echo HTTP ACCEPT = "S$HTTP ACCEPT" 

47 echo PATH INFO = "SPATH INFO" 

48 echo PATH TRANSLATED = "SPATH TRANSLATED" 

49 echo SCRIPT NAME = "SSCRIPT NAME" 

50 echo QUERY STRING = "SQUERY STRING" 

51 echo REMOTE HOST = S$REMOTE HOST 


52 echo REMOTE_ADDR = $REMOTE_ADDR 
53 echo REMOTE_USER = $REMOTE_USER 
54A echo AU LHe EH = SENOMDSE TYPE, 
55 echo CONTENT TYPE = SCONTENT_TYPE 
56 echo CONTENT LENGTH = $CONTENT LENGTH 


58 exit 0 


60 # Here document to give short instructions. 
Gil ge" tebr (COE V 


63 1L) Drop Emis alin your Incicjos / /clhomanin meme/eoisloilin chimeacicoimy o 
GA 2) Them, Gben arto: // comen. memel egion TESE Copal c Sal e 


66 test CGI. 
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Appendix R. Copyright 


The Advanced Bash Scripting Guide is copyright © 2000, by Mendel Cooper. [1] The author also asserts 
copyright on all previous versions of this document. [2] 


This blanket copyright recognizes and protects the rights of all contributors to this document. 


This document may only be distributed subject to the terms and conditions set forth in the Open Publication 
License (version 1.0 or later), http://www.opencontent.org/openpub/. The following license options also 


apply. 


1A. Distribution of substantively modified versions of this document 
2 is permitted only under the following provisions. 
3 
4 Al. The modified document must clearly indicate that it is derivative 
E of the original Advanced Bash Scripting Guide, and the original 
6 author, Mendel Cooper, must be listed as the primary author. 
7 
8 A2. The modified or derivative document must clearly indicate which portions 
9 of the text differ or deviate from the original document. A notice must 
10 be present, stating that the original author does not necessarily 
JL ndorse the changes to the original. 
LZ 
13 A3. The modified or derivative document must be distributed under this 
14 same license, and the original author's copyright, as applicable, 
15 may not be modified. 
LG 
17 
18 B. This document, or any modified or derivative version thereof, may 
Lg NOT be distributed encrypted or with any form of DRM (Digital Rights 
20 Management) or content-control mechanism embedded in it. Nor may this 
21 document or any derivative thereof be bundled with other DRM-ed works. 
BE 
23 € If this document (or any previous version or derivative thereof) 
24 is made available on a Web or ftp site, then the file(s) must be 
25 publicly accessible. No password or other access restrictions to 
26 its download may be imposed. 
2 
AS JD) Distribution of the original work in any standard (paper) book form 
29 requires permission from the copyright holder. 
30 
Sil dm In the event that the author or maintainer of this document cannot 
32 be contacted, the Linux Documentation Project is authorized to 
ES take over custodianship of the document and name a new maintainer, 
34 who would then have the right to update and modify the document. 


Without explicit written permission from the author, distributors and publishers (including on-line publishers) 
are prohibited from imposing any additional conditions, strictures, or provisions on this document, any 
previous versions, or any derivative versions. As of this update, the author asserts that he has not entered into 
any contractual obligations that would alter the foregoing declarations. 


Essentially, you may freely distribute this book or any derivative thereof in electronic form. 
If you display or distribute this document, any previous versions thereof, or any derivatives thereof under any 
license except the one above, then you are required to obtain the author's written permission. Failure to do so 


may terminate your distribution rights. 


Additionally, the following waiver applies: 


1 By copying or distributing this book you WAIVE THE RIGHT 


to use the materials within, or any portion thereof, in a patent or copyright 
lawsuit against the Open Source community, its developers, its 

distributors, or against any of its associated software or documentation 
including, but not limited to, the Linux kernel, Open Office, Samba, 

and Wine. You further WAIVE THE RIGHT to use any of the materials within 

this book in testimony or depositions as a plaintiff's "expert witness" in 
any lawsuit against the Open Source community, any of its developers, its 

9 distributors, or any of its associated software or documentation. 


These are very liberal terms, and they should not hinder any legitimate distribution or use of this book. The 
author especially encourages its use for classroom and instructional purposes. 


co —10) 0! 4 C) ND 


£^) Certain of the scripts contained in this document are, where noted, in the Public Domain. These scripts 
are exempt from the foregoing license and copyright restrictions. 
The commercial print and other rights to this book are available. Please contact the author if interested. 


The author produced this book in a manner consistent with the spirit of the LDP Manifesto. 


Linux is a trademark registered to Linus Torvalds. 

Fedora is a trademark registered to Red Hat. 

Unix and UNIX are trademarks registered to the Open Group. 
MS Windows is a trademark registered to the Microsoft Corp. 
Solaris is a trademark registered to Sun, Inc. 

OSX is a trademark registered to Apple, Inc. 

Yahoo is a trademark registered to Yahoo, Inc. 

Pentium is a trademark registered to Intel, Inc. 

Thinkpad is a trademark registered to Lenovo, Inc. 


Scrabble is a trademark registered to Hasbro, Inc. 


Librie, PRS-500, and PRS-505 are trademarks registered to Sony, Inc. 


All other commercial trademarks mentioned in the body of this work are registered to their respective 
owners. 


Hyun Jin Cha has done a Korean translation of version 1.0.11 of this book. Spanish, Portuguese, French, 
German, Italian, Russian, Czech, Chinese, Indonesian, and Dutch translations are also available or in progress. 
If you wish to translate this document into another language, please feel free to do so, subject to the terms 
stated above. The author wishes to be notified of such efforts. 


Notes 


[1] A printed edition of a substantively rewritten version of the book will be released in fall of 2010. 

[2] The author intends that this book be released into the Public Domain after a period of 14 years from 
initial publication, that is, in 2014. In the early years of the American republic this was the duration 
statutorily granted to a copyrighted work. 
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Appendix S. ASCII Table 


In a book of this sort it is traditional to have an ASCII Table appendix. This book does not. Instead, here is a 
short shell script that generates a complete ASCII table and writes it to the file ASCII.txt. 


Example S-1. A script that generates an ASCII table 


1 #!/bin/bash 

2 $^ meus eim 

3 # ver. 0.2, reldate 26 Aug 2008 

4 # Patched by ABS Guide author. 

5 

6 # Original script by Sebastian Arming. 

7 # Used with permission (thanks!). 

8 

J eree SUNSICOLIS EZE io Seye Sicclowie Teo. Cile, 

10 #+ as in the example scripts 

dal #+ reassign-stdout.sh and upperconv.sh. 
1,2 

13 MAXNUM-256 

14 COLUMNS-5 

15 OCT-8 

16 OCTSQU-64 

17 LitTLESPACE=—3 

18 BIGSPACE--5 

19 
20 i-1 # Decimal counter 
21 o-1 # Octal counter 

22 

AS wanlla | VWSsw sake "SWBOgWONN IB Ck We don't have to count past 400 octal. 
24 paddi=" gu 

235 echo -n "$(paddi: SBIGSPACE} " # Column spacing. 
26 paddo-"00$o" 

27 # echo -ne "\\${paddo: SLITTLESPACE}" # Original. 
28 echo -ne "\\0${paddo: $LITTLESPACE)" # Fixup. 

29 # $ 

30) echo -n " T" 

Sil if (( i % SCOLUMNS == 0)); then # New line. 
32 echo 

33 i3 

34 ( ip arte) }) 

S3 ur idee Geral norec iog Cor 3} abe 10, awe 64 cecina is 100) 0era, 
36 (( i $ SOCT == 0)) && ((O+=2) ) 

31 (( i $ SOCTSQU == 0)) && ((0o*-20)) 

38 done 

39 

AQ) ez GE 

41 


42 # Compare this script with the "pr-asc.sh" example. 
43 # This one handles "unprintable" characters. 


45 4 Exercise: 
46 # Rewrite this script to use decimal numbers, rather than octal. 
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Index 


This index / glossary / quick-reference lists many of the important topics covered in the text. Terms are 
arranged in approximate ASCII sorting order, modified as necessary for enhanced clarity. 


Note that commands are indexed in Part 4. 
ok ook ok 
^ (caret) 


* Beginning-of-line, in a Regular Expression 


e^ 
AN 


Uppercase conversion in parameter substitution 


~ Tilde 


e ~ home directory, corresponds to SHOME 
e ~/ Current user's home directory 

e ~+ Current working directory 

e ~- Previous working directory 


= Equals sign 


e = Variable assignment operator 
e = String comparison operator 


== String comparison operator 
e =~ Regular Expression match operator 


Example script 
< Left angle bracket 
e [s-less-than 
String comparison 


Integer comparison within double parentheses 
* Redirection 


« stdin 

«« Here document 

««« Here string 

«» Opening a file for both reading and writing 


» Right angle bracket 


* [s-greater-than 


String comparison 


Integer comparison, within double parentheses 
* Redirection 


> Redirect stdout to a file 

>> Redirect stdout to a file, but append 

i» &j Redirect file descriptor i to file descriptor j 

>&j Redirect stdout to file descriptor j 

>&2 Redirect stdout of acommand to stderr 

2>&1 Redirect stderr to stdout 

&> Redirect both stdout and stderr of a command to a file 

:> file Truncate file to zero length 
| Pipe, a device for passing the output of a command to another command or to the shell 
Il Logical OR test operator 
- (dash) 

e Prefix to default parameter, in parameter substitution 


* Prefix to option flag 
* Indicating redirection from stdin or stdout 


e -- (double-dash) 


Prefix to Jong command options 
C-style variable decrement within double parentheses 


; (semicolon) 


* As command separator 
* S Escaped semicolon, terminates a find command 
e ;; Double-semicolon, terminator in a case option 


Required when ... 


do keyword is on the first line of loop 


terminating curly-bracketed code block 
© ;;& ;& Terminators in a case option (version 4+ of Bash). 


: Colon, null command, equivalent to the true Bash builtin 


e :» file Truncate file to zero length 


! Negation operator, inverts exit status of a test or command 
* != not-equal-to String comparison operator 
? (question mark) 


e Match zero or one characters, in an Extended Regular Expression 


e Single-character wild card, in globbing 
* [n a C-style Trinary operator 


// Double forward slash, behavior of cd command toward 
. (dot / period) 


* . Load a file (into a script), equivalent to source command 


e . Match single character, in a Regular Expression 
e . Current working directory 


J Current working directory 
e .. Parent directory 


... ' (single quotes) strong quoting 


... '' (double quotes) weak quoting 


© Double-quoting the backslash (X) character 


* Comma operator 


e, 


29 
Lowercase conversion in parameter substitution 
( Parentheses 
*(...) Command group; starts a subshell 
* (...) Enclose group of Extended Regular Expressions 
e »(...) 
«( ... ) Process substitution 
e ...) Terminates test-condition in case construct 
* (( ... )) Double parentheses, in arithmetic expansion 


[ Left bracket, test construct 


[ ]Brackets 


* Array element 
* Enclose character set to match in a Regular Expression 


* Test construct 


[[ -.. ]] Double brackets, extended test construct 
$ Anchor, in a Regular Expression 
$ Prefix to a variable name 
$( ...) Command substitution, setting a variable with output of a command, using parentheses notation 
^.^ Command substitution, using backquotes notation 
$I ... ] Integer expansion (deprecated) 
${ ... ) Variable manipulation / evaluation 
e ${var} Value of a variable 
e ${#var} Length of a variable 


° S{#@} 


${#*} Number of positional parameters 
* ${parameter?err_msg} Parameter-unset message 
* ${parameter-default} 


${parameter:-default} 
${parameter=default} 


${parameter:=default} Set default parameter 
e ${parameter+alt_value} 


${parameter:+alt_value} 


Alternate value of parameter, if set 


e ${!var} 


Indirect referencing of a variable, new notation 
© ${!#} 


Final positional parameter. (This is an indirect reference to Sit.) 
e ${!varprefix*} 


${!varprefix @} 


Match names of all previously declared variables beginning with varprefix 
* ${string:position} 


${string:position:length} Substring extraction 
e ${var#Pattern} 


${var##Pattern} Substring removal 
* ${var % Pattern} 


${var % % Pattern} Substring removal 
* ${string/substring/replacement} 


${string//substring/replacement} 
${string/#substring/replacement} 
${string/% substring/replacement} Substring replacement 
\ Escape the character following 
* \<... V Angle brackets, escaped, word boundary in a Regular Expression 
e\{ N \} "Curly" brackets, escaped, number of character sets to match in an Extended RE 
e V Semicolon, escaped, terminates a find command 


e \$$ Indirect reverencing of a variable, old-style notation 
* Escaping a newline, to write a multi-line command 


* &» Redirect both stdout and stderr of a command to a file 
* >&j Redirect stdout to file descriptor j 


>&2 Redirect stdout of a command to stderr 


* ji» &j Redirect file descriptor i to file descriptor j 


2>&1 Redirect stderr to stdout 
* Closing file descriptors 


n<&- Close input file descriptor n 
0<&-, <&- Close stdin 
n>&- Close output file descriptor n 
1>&-, >&- Close stdout 
* && Logical AND test operator 
* Command & Run job in background 
# Hashmark, special symbol beginning a script comment 
#! Sha-bang, special string starting a shell script 
* Asterisk 
* Wild card, in globbing 
* Any number of characters in a Regular Expression 
e ** Exponentiation, arithmetic operator 
e ** Extended globbing file-match operator 


% Percent sign 


* Modulo, division-remainder arithmetic operation 
e Substring removal (pattern matching) operator 


+ Plus sign 


* Character match, in an extended Regular Expression 


e Prefix to alternate parameter, in parameter substitution 
* ++ C-style variable increment, within double parentheses 


ok ook ok 
Shell Variables 


$_ Last argument to previous command 
$- Flags passed to script, using set 


$! Process ID of last background job 


$? Exit status of a command 

$@ All the positional parameters, as separate words 
$* All the positional parameters, as a single word 
$$ Process ID of the script 


$# Number of arguments passed to a function, or to the script itself 


$0 Filename of the script 
$1 First argument passed to script 
$9 Ninth argument passed to script 
Table of shell variables 
Kook cK OK k k 
-a Logical AND compound comparison test 
Address database, script example 
Advanced Bash Scripting Guide, where to download 
Alias 

e Removing an alias, using unalias 
Anagramming 
And list 

* To supply default command-line argument 
And logical operator && 


Angle brackets, escaped, \< . . . V» word boundary in a Regular Expression 


Anonymous Aere document, using : 


Archiving 


* rpm 
* tar 


Arithmetic expansion 


* variations of 


Arithmetic operators 


* combination operators, C-style 


+= -= *- /= VES 


G In certain contexts, += can also function as a string concatenation operator. 


Arrays 


* Associative arrays 

* Bracket notation 

* Concatenating, example script 
* Copying 

* Declaring 


declare -a array name 


* Embedded arrays 


* Empty arrays, empty elements, example script 
* [ndirect references 


* [nitialization 


array-( elementl element2 ... elementN) 


Example script 


Using command substitution 
* Loading a file into an array 
* Multidimensional, simulating 
* Nesting and embedding 


* Notation and usage 
e Number of elements in 


S(farray name[G8]) 


S(farray name[*]) 


* Operations 

* Passing an array to a function 

* As return value from a function 
e Special properties, example script 
e String operations, example script 
* unset deletes array elements 


Arrow keys, detecting 


ASCII table 
awk field-oriented text processing language 


* rand(), random function 


e String manipulation 
* Using export to pass a variable to an embedded awk script 


ok ck ok 


Backquotes, used in command substitution 
Base conversion, example script 


Bash 


* Bad scripting practices 
* Basics reviewed, script example 


e Command-line options 


Table 
* Features that classic Bourne shell lacks 
* [nternal variables 
* Version 2 
* Version 3 
* Version 4 


.bashrc 


BASH SUBSHELL 


Basic commands, external 
Batch files, DOS 

Batch processing 

bc, calculator utility 


e In a here document 
* Template for calculating a script variable 


Bibliography 
Bison utility 
Bitwise operators 


Block devices 


e testing for 


Blocks of code 


* Redirection 
Script example: redirecting output of a a code block 


Brace expansion 


e Extended, {a..z} 
e With increment and zero-padding (new feature in Bash, version 4) 


Brackets, [ ] 


* Array element 
* Enclose character set to match in a Regular Expression 
* Test construct 


Brackets, curly, {}, used in 


* Code block 
° find 
* Extended Regular Expressions 
© Positional parameters 
* xargs 
break /oop control command 


* Parameter (optional) 


Builtins in Bash 


* Do not fork a subprocess 


case construct 


* Command-line parameters, handling 
e Globbing, filtering strings with 


cat, concatentate file(s) 


* Abuse of 
* cat scripts 


* Less efficient than redirecting stdin 


* Piping the output of, to a read 
* Uses of 


Character devices 


e testing for 


Checksum 


Child processes 


Colon, : , equivalent to the true Bash builtin 


Colorizing scripts 


* Cycling through the background colors, example script 
e Table of color escape sequences 
* Template, colored text on colored background 


Comma operator, linking commands or operations 


Command-line options 


command not found handle () builtin error-handling function (version 4+ of Bash) 
Command substitution 


e $( ... ), preferred notation 


* Backquotes 


* Extending the Bash toolset 
* [nvokes a subshell 


* Nesting 

* Removes trailing newlines 

e Setting variable from loop output 
* Word splitting 


Comment headers, special purpose 


Commenting out blocks of code 


* Using an anonymous here document 
e Using an if-then construct 


Communications and hosts 


Compound comparison operators 
Compression utilities 


* bzip2 
* compress 
© ozi 


e zip 
continue loop control command 
Control characters 


e Control-C, break 

* Control-D, terminate / log out / erase 
* Control-G, BEL (beep) 

e Control-H, rubout 

* Control-J, newline 

* Control-M, carriage return 


Coprocesses 


cron, scheduling daemon 


C-style syntax , for handling variables 


Crossword puzzle solver 


Curly brackets { } 


e in find command 
e in an Extended Regular Expression 
* in xargs 


Daemons, in UNIX-type OS 
date 

dc, calculator utility 

dd, data duplicator command 


* Conversions 
* Copying raw data to/from devices 
e File deletion, secure 


e Keystrokes, capturing 

* Options 

* Random access on a data stream 

e Swapfiles, initializing 

* Thread on www.linuxquestions.org 


Debugging scripts 


* Tools 

e Trapping at exit 

e Trapping signals 
Decimal number, Bash interprets numbers as 
declare builtin 

* options 

case-modification options (version 4+ of Bash) 

Default parameters 
/dev directory 

e /dev/null pseudo-device file 


e /dev/urandom pseudo-device file, generating pseudorandom numbers with 
e / dev/ zero, pseudo-device file 


Device file 


dialog, utility for generating dialog boxes in a script 


SDIRSTACK directory stack 
Disabled commands, in restricted shells 
do keyword, begins execution of commands within a loop 
done keyword, terminates a loop 
DOS batch files, converting to shell scripts 
DOS commands, UNIX equivalents of (table) 
dot files, "hidden" setup and configuration files 
Double brackets [[ ... ]] test construct 
e and evaluation of octal/hex constants 


Double parentheses (( ... )) arithmetic expansion/evaluation construct 


Double quotes " ... ' weak quoting 
* Double-quoting the backslash (X) character 


Double-spacing a text file, using sed 


ok ck ok 


-e File exists test 


echo 


e Feeding commands down a pipe 
e Setting a variable using command substitution 


e /bin/echo, external echo command 
elif, Contraction of else and if 
else 
Encrypting files, using openssl 
esac, keyword terminating case construct 


Environmental variables 


-eq , is-equal-to integer comparison test 


Eratosthenes, Sieve of, algorithm for generating prime numbers 


Escaped characters, special meanings of 
/ etc/fstab (filesystem mount) file 


/ etc/passwd (user account) file 


SEUID, Effective user ID 


eval, Combine and evaluate expression(s), with variable expansion 
e Effects of, Example script 


* Forces reevaluation of arguments 
e And indirect references 


* Risk of using 
* Using eval to select among variables 


Evaluation of octal/hex constants within [[ ... 
exec command, using in redirection 
Exercises 

Exit and Exit status 


* exit command 
* Exit status (exit code, return status of a command) 


Table, Exit codes with special meanings 

Out of range 

Pipe exit status 

Specified by a function return 

Successful, 0 

/usr/inciude/sysexits.h, system file listing C/C++ standard exit codes 


Export, to make available variables to child processes 


* Passing a variable to an embedded awk script 


expr, Expression evaluator 


e Substring extraction 
e Substring index (numerical position in string) 
e Substring matching 


Extended Regular Expressions 


e ? (question mark) Match zero / one characters 


e (...) Group of expressions 
eX NV "Curly" brackets, escaped, number of character sets to match 


* 4- Character match 


x k ok 
factor, decomposes an integer into its prime factors 
e Application: Generating prime numbers 
false, returns unsuccessful (1) exit status 
Field, a group of characters that comprises an item of data 
Files / Archiving 
File descriptors 
e Closing 
n<&- Close input file descriptor n 
0<&-, <&- Close stdin 
n>&- Close output file descriptor n 


1>&-, >&- Close stdout 
e File handles in C, similarity to 


File encryption 


find 

* {} Curly brackets 

* S Escaped semicolon 
Filter 


* Using - with file-processing utility as a filter 
* Feeding output of a filter back to same filter 


Eloating point numbers, Bash does not recognize 

fold, a filter to wrap lines of text 

Forking a child process 

for loops 

Eunctions 
* Arguments passed referred to by position 
* Capturing the return value of a function using echo 
* Definition must precede first call to function 


* Exit status 
* Local variables 


and recursion 
e Passing an array to a function 
* Passing pointers to a function 
* Positional parameters 
e Recursion 
* Redirecting stdin of a function 
* return 


Multiple return values from a function, example script 
Returning an array from a function 


return range limits, workarounds 
e shift arguments passed to a function 


Games and amusements 


e Anagrams 

e Anagrams, again 

* Crossword puzzle solver 
* Crypto-Quotes 


* Dealing a deck of cards 
* Fifteen Puzzle 


* Horse race 

* Knight's Tour 

* "Life" game 

* Magic Squares 

* Music-playing script 
e Nim 

* Pachinko 


* Perquackey 
* Petals Around the Rose 


* Podcasting 
* Poem 


* Towers of Hanoi 


Graphic version 
Alternate graphic version 


getopt, external command for parsing script command-line arguments 


* Emulated in a script 


getopts, Bash builtin for parsing script command-line arguments 


e. SOPTIND / SOPTARG 
Global variable 


Globbing, filename expansion 


* Wild cards 
e Will not match dot files 


Golden Ratio (Phi) 


-ge , greater-than or equal integer comparison test 


-gt , greater-than integer comparison test 


groff, text markup and formatting language 
SGROUPS, Groups user belongs to 
gzip, compression utility 
ok ok k 
Hashing, creating lookup keys in a table 
e Example script 
head, echo to stdout lines at the beginning of a text file 


help, gives usage summary of a Bash builtin 


Here documents 


e Anonymous here documents, using : 


Commenting out blocks of code 


Self-documenting scripts 
* bc in a here document 


* cat scripts 

* Command substitution 

* ex scripts 

e Function, supplying input to 
* Here strings 


Calculating the Golden Ratio 
Prepending text 


Using read 
e Limit string 


! as a limit string 
Closing limit string may not be indented 
Dash option to limit string, ««-LimitString 


e Literal text output, for generating program code 
* Parameter substitution 


Disabling parameter substitution 
* Passing parameters 
* Temporary files 
* Using vi non-interactively 


History commands 


HOME, user's home directory 


Homework assignment solver 


SHOSTNAME, system host name 


ok ck ck 


$Id parameter, in rcs (Revision Control System) 
if [ condition ]: then ... test construct 
e if-grep, if and grep in combination 


Fixup for if-grep test 


SIFS, Internal field separator variable 


* Defaults to whitespace 
Integer comparison operators 
in, keyword preceding [list] in a for loop 
Initialization table, /etc/inittab 
Inline group, i.e., code block 


Interactive script, test for 


I/O redirection 
Indirect referencing of variables 
* New notation, introduced in version 2 of Bash ( example script) 


Iteration 


Job IDs, table 
jot, Emit a sequence of integers. Equivalent to seq. 


e Random sequence generation 


x cK ck 


Keywords 
kill, terminate a process by process ID 
e Options (-1, -9) 
killall, terminate a process by name 
killall script in /etc/rc.d/init.d 
ok cK ok 
-le , less-than or equal integer comparison test 
let, setting and carrying out arithmetic operations on variables 


* C-style increment and decrement operators 


Limit string, in a here document 


LINENO, variable indicating the line number where it appears in a script 


Link, file (using /n command) 


* Invoking script with multiple names, using /n 
e symbolic links, In -s 


List constructs 


e And list 
* Or list 


Local variables 

* and recursion 
Localization 
Logical operators (&&, | |, etc.) 
Logout file, the ~/ .bash_logout file 
Loopback device, mounting a file on a block device 
Loops 

* break loop control command 


* continue loop control command 
* C-style loop within double parentheses 


for loop 


while loop 
* do (keyword), begins execution of commands within a loop 


* done (keyword), terminates a loop 


e for loops 


forarg in [list];do 

Command substitution to generate [1ist 

Filename expansion in [list 

Multiple parameters in each [1ist] element 
Omitting [list], defaults to positional parameters 


Parameterizing [list 


Redirection 
e in, (keyword) preceding [list] in a for loop 
* Nested loops 


e Running a loop in the background, script example 
e Semicolon required, when do is on first line of loop 


for loop 


while loop 
e until loop 


until [ condition-is-true ]; do 
* while loop 


while [ condition ]; do 


Function call inside test brackets 


Multiple conditions 


Omitting test brackets 


Redirection 


while read construct 
* Which type of loop to use 


Loopback devices 


* [n / dev directory 
* Mounting an ISO image 


-lt , less-than integer comparison test 


x cK ok 


m4, macro processing language 


SMACHTYPE, Machine type 


Magic number, marker at the head of a file indicating the file type 
Makefile, file containing the list of dependencies used by make command 
man, manual page (lookup) 
e Man page editor (script) 
mapfile builtin, loads an array with a text file 
Math commands 
Meta-meaning 
Modulo, arithmetic remainder operator 
* Application: Generating prime numbers 
Mortgage calculations, example script 


ok ck ok 


-n String not null test 


Named pipe, a temporary FIFO buffer 

e Example script 
nc, netcat, a toolkit for TCP and UDP ports 
-ne, not-equal-to integer comparison test 
Negation operator, !, reverses the sense of a test 
netstat, Network statistics 
nl, a filter to number lines of text 


Noclobber, -C option to Bash to prevent overwriting of files 


NOT logical operator, ! 


null variable assignment, avoiding 


ok ck ok 


-0 Logical OR compound comparison test 
octal, base-8 numbers 
od, octal dump 


SOLDPWD Previous working directory 


openssl encryption utility 
Operator 


e Definition of 
* Precedence 


Options, passed to shell or script on command line or by set command 


Or list 


Or logical operator, ll 


ok ck ck 


Parameter substitution 
e ${parameter+alt_value} 
${parameter: alt. value] 


Alternate value of parameter, if set 
© ${parameter-default} 


${parameter: -default} 
${parameter=default} 
${parameter:=default} 


Default parameters 
e ${!varprefix*} 


${!varprefix@ } 


Parameter name match 
e $/parameter?err. msg] 


Parameter-unset message 
e ${parameter} 


Value of parameter 
© Case modification (version 4+ of Bash). 


e Script example 
* Table of parameter substitution 


Parent / child process problem, a child process cannot export variables to a parent process 


Parentheses 


* Command group 
* Enclose group of Extended Regular Expressions 


* Double parentheses, in arithmetic expansion 


SPATH, the path (location of system binaries) 
Perl, programming language 


* Combined in the same file with a Bash script 
* Embedded in a Bash script 


Perquackey-type anagramming game (Quackey script) 


Petals Around the Rose 
PID, Process ID, an identification number assigned to a running process. 


Pipe, | , a device for passing the output of a command to another command or to the shell 


* Avoiding unnecessary commands in a pipe 
e Comments embedded within 


* Exit status of a pipe 
* Pipefail, set -o pipefail option to indicate exit status within a pipe 
e SPIPESTATUS, exit status of last executed pipe 


* Piping output of a command to a script 
e Redirecting st din, rather than using cat in a pipe 


Pitfalls 


* - (dash) is not redirection operator 
* // (double forward slash), behavior of cd command toward 


e #!/bin/sh script header disables extended Bash features 
* Abuse of cat 

* CGI programming, using scripts for 

* Closing limit string in a here document, indenting 

e DOS-type newlines (\r\n) crash a script 

© Double-quoting the backslash (X) character 


* eval, risk of using 

e Execute permission lacking for commands within a script 

* Export problem, child process to parent process 

e Extended Bash features not available 

e Failing to quote variables within test brackets 

e GNU command set, in cross-platform scripts 

* let misuse: attempting to set string variables 

* Multiple echo statements in a function whose output is captured 
* null variable assignment 


e Numerical and string comparison operators not equivalent 

= and -eq not interchangeable 
* Omitting terminal semicolon, in a curly-bracketed code block 
e Piping 

echo to a loop 


echo to read (however, this problem can be circumvented) 


tail -£ to grep 
* Preserving whitespace within a variable, unintended consequences 


© suid commands inside a script 

* Undocumented Bash features, danger of 
* Uninitialized variables 

* Variable names, inappropriate 

* Variables in a subshell, scope limited 


e Subshell in while-read loop 
e Whitespace, misuse of 


Pointers 


* and file descriptors 
* and functions 


* and indirect references 


* and variables 


Portability issues in shell scripting 


e Setting path and umask 
* A test suite script (Bash versus classic Bourne shell) 


e Using whatis 


Positional parameters 


e SQ, as separate words 
e $>, as a single word 
* in functions 


POSIX, Portable Operating System Interface / UNIX 
e —posix option 


e 1003.2 standard 
* Character classes 


SPPID, process ID of parent process 
Precedence, operator 
Prepending lines at head of a file, script example 


Prime numbers 


* Generating primes using the factor command 
e Generating primes using the modulo operator 


e Sieve of Eratosthenes, example script 
printf, formatted print command 


/ proc directory 


e Running processes, files describing 
* Writing to files in /proc, warning 


Process 


* Child process 


* Parent process 
* Process ID (PID) 


Process substitution 


* To compare contents of directories 
e To supply st din of a command 


* Template 
* while-read loop without a subshell 


Programmable completion (tab expansion) 


Prompt 


e SPS1, Main prompt, seen at command line 
e SPS2, Secondary prompt 


Pseudo-code, as problem-solving method 


SPWD, Current working directory 


kk OK 


Quackey, a Perquackey-type anagramming game (script) 
Question mark, ? 


* Character match in an Extended Regular Expression 


e Single-character wild card, in globbing 
* [n a C-style Trinary operator 


Quoting 


* Character string 
* Variables 


within fest brackets 
e Whitespace, using quoting to preserve 


kk OK 


Random numbers 


e /dev/urandom 

* rand (), random function in awk 

e SRANDOM, Bash function that returns a pseudorandom integer 
e Random sequence generation, using date command 

e Random sequence generation, using jot 

* Random string, generating 


ICS 


read, set value of a variable from st din 


* Detecting arrow keys 

* Options 

* Piping output of cat to read 

e "Prepending" text 

* Problems piping echo to read 
* Redirection from a file to read 
e SREPLY, default read variable 
e Timed input 

e while read construct 


readline library 
Recursion 


* Demonstration of 
* Factorial 


* Fibonacci sequence 
* Local variables 


e Script calling itself recursively 
* Towers of Hanoi 


Redirection 


* Code blocks 
e exec «filename, 


to reassign file descriptors 


e [ntroductory-level explanation of I/O redirection 
* Open a file for both reading and writing 


<>filename 
* read input redirected from a file 
e stderr to stdout 


2>&1 
e stdin/ stdout, using - 


e stdinof a function 


e stdout to a file 


2.4.22 


* stdout to file descriptor j 


2&]j 

* file descriptori to file descriptor j 
i»&j 

e stdout of a command to stderr 


>&2 
e stdout and stderr of a command to a file 


&> 
* tee, redirect to a file output of command(s) partway through a pipe 


Reference Cards 


* Miscellaneous constructs 


* Parameter substitution/expansion 
e Special shell variables 


e String operations 
e Test operators 


Binary comparison 


Files 
Regular Expressions 


* ^ (caret) Beginning-of-line 
e $ (dollar sign) Anchor 
e . (dot) Match single character 


e * (asterisk) Any number of characters 
e [ ] (brackets) Enclose character set to match 


o \ (backslash) Escape, interpret following character literally 
e \< ... V (angle brackets, escaped) Word boundary 
* Extended REs 


+ Character match 


M \} Escaped "curly" brackets 


[: :] POSIX character classes 


SREPLY, Default value associated with read command 
Restricted shell, shell (or script) with certain commands disabled 
return, command that terminates a function 
run-parts 

e Running scripts in sequence, without user intervention 
x k ok 
Scope of a variable, definition 
Script options, set at command line 
Scripting routines, library of useful definitions and functions 
Secondary prompt, $PS2 


Security issues 


e nmap, network mapper / port scanner 
* sudo 
© suid commands inside a script 


* Viruses, trojans, and worms in scripts 
* Writing secure scripts 


sed, pattern-based programming language 


* Table, basic operators 
e Table, examples of operators 


select, construct for menu building 
ein listomitted 
Semaphore 
Semicolon required, when do keyword is on first line of loop 
* When terminating curly-bracketed code block 
seg, Emit a sequence of integers. Equivalent to jot. 
set, Change value of internal script variables 
Shell script, definition of 
Shell wrapper, script embedding a command or utility 
shift, reassigning positional parameters 
SSHLVL, shell level, depth to which the shell (or script) is nested 
shopt, change shell options 
Signal, a message sent to a process 
Simulations 
* Brownian motion 


e Galton board 
* Horserace 


e Life, game of 
e PI, approximating by firing cannonballs 
* Pushdown stack 


Single quotes (' ... ') strong quoting 
Socket, a communication node associated with an I/O port 
Sorting 


e Bubble sort 
* [nsertion sort 


source, execute a script or, within a script, import a file 


* Passing positional parameters 


Spam, dealing with 


e Example script 
e Example script 
e Example script 
e Example script 


Special characters 


Stack 


* Definition 
e Emulating a push-down stack, example script 


Standard Deviation, example script 


Startup files, Bash 


stdinand stdout 
Stopwatch, example script 
Strings 


e =~ String match operator 
* Comparison 
* Length 


S{#string} 
* Manipulation 
* Manipulation, using awk 
* Null string, testing for 
* Protecting strings from expansion and/or reinterpretation, script example 


Unprotecting strings, script example 
e strchr(), equivalent of 


e strlen(), equivalent of 
e strings command, find printable strings in a binary or data file 


e Substring extraction 
S string:position] 
${string:position:length} 
Using expr 
e Substring index (numerical position in string) 
e Substring matching, using expr 
e Substring removal 
${var#Pattern} 


${var##Pattern} 


var%Pattern 
${var%%Pattern } 


e Substring replacement 


${string/substring/replacement} 
${string//substring/replacement} 
${string/#substring/replacement} 
${string/%substring/replacement} 


Script example 
e Table of string/substring manipulation and extraction operators 


' 


Strong quoting ' ... 


Stylesheet for writing scripts 


Subshell 


* Command list within parentheses 
e Variables, SBASH_SUBSHELL and SSHLVL 


e Variables in a subshell 
scope limited, but ... 
... can be accessed outside the subshell? 
su Substitute user, log on as a different user or as root 


suid (set user id) file flag 


© suid commands inside a script, not advisable 
Symbolic links 
Swapfiles 


ok ck ok 


Tab completion 


Table lookup, script example 

tail, echo to stdout lines at the (tail) end of a text file 

tar, archiving utility 

tee, redirect to a file output of command(s) partway through a pipe 


Terminals 


* setserial 
* setterm 
* stt 


e tput 
* wall 


test command 


e Bash builtin 
e external command, /usr/bin/test (equivalent to /usr/bin/ [) 


Test constructs 
Test operators 


* -a Logical AND compound comparison 
e -e File exists 

* -eq is-equal-to (integer comparison) 

e -f File is a regular file 


e -ge greater-than or equal (integer comparison) 
* -gt greater-than (integer comparison) 


e -le less-than or equal (integer comparison) 
e -It less-than (integer comparison) 

* -n not-zero-length (string comparison) 

* -ne not-equal-to (integer comparison) 

* -0 Logical OR compound comparison 

e -u suid flag set, file test 

e -Z is-zero-length (string comparison) 

e = is-equal-to (string comparison) 


== is-equal-to (string comparison) 
* < less-than (string comparison) 
e < less-than, (integer comparison, within double parentheses) 
e <= less-than-or-equal, (integer comparison, within double parentheses) 
* » greater-than (string comparison) 
* » greater-than, (integer comparison, within double parentheses) 
e >= greater-than-or-equal, (integer comparison, within double parentheses) 
* || Logical OR 
* && Logical AND 
* ! Negation operator, inverts exit status of a test 


!= not-equal-to (string comparison) 
e Tables of test operators 


Binary comparison 
File 
Text and text file processing 
Time / Date 


Timed input 


* Using read -t 

e Using stty 

* Using timing loop 
e Using STMOUT 


Tips and hints for Bash scripts 


* Array, as return value from a function 
* Capturing the return value of a function, using echo 
* Comment blocks 


Using anonymous here documents 


Using if-then constructs 


e Comment headers, special purpose 

* C-style syntax , for manipulating variables 
* Double-spacing a text file 

* Filenames prefixed with a dash, removing 
e Filter, feeding output back to same filter 

e Function return value workarounds 

* if-grep test fixup 

e Library of useful definitions and functions 
* null variable assignment, avoiding 


e Passing an array to a function 
e Prepending lines at head of a file 


* Progress bar template 

* Pseudo-code 

* rcs 

e Redirecting a fest to / dev/nul11 to suppress output 


e Running scripts in sequence without user intervention, using run-parts 
e Script as embedded command 


e Script portability 
Setting path and umask 


Using whatis 
e Setting script variable to a block of embedded sed or awk code 


e Subshell variable, accessing outside the subshell 
e Testing a variable to see if it contains only digits 


e Testing whether a command exists, using type 


e Tracking script usage 
* while-read loop without a subshell 


* Widgets, invoking from a script 
STMOUT, Timeout interval 
Token, a symbol that may expand to a keyword or command 
tput, terminal-control command 
tr, character translation filter 


e DOS to Unix text file conversion 
* Options 


e Soundex, example script 
* Variants 


Trap, specifying an action upon receipt of a signal 
Trinary operator, C-style, var»10?88:99 


e in double-parentheses construct 
* in let construct 


true, returns successful (0) exit status 


typeset builtin 


* options 


ok ck ok 


SUID, User ID number 

unalias, to remove an alias 

uname, output system information 

Uninitialized variables 

uniq, filter to remove duplicate lines from a sorted file 
unset, delete a shell variable 

until loop 

until [ condition-is-true ]; do 

xk k 


Variables 


e Array operations on 
* Assignment 


Script example 
Script example 


Script example 
* Bash internal variables 


* Block of sed or awk code, setting a variable to 


* C-style increment/decrement/trinary operations 
* Change value of internal script variables using set 
* declare, to modify the properties of variables 


* Deleting a shell variable using unset 
* Environmental 


* Expansion / Substring replacement operators 


* [ndirect referencing 


eval variablel=\SSvariable2 
Newer notation 


S{!ivariable} 
* Integer 
* Integer / string (variables are untyped) 


* Length 


S{#var} 
e Lvalue 
* Manipulating and expanding 
* Name and value of a variable, distinguishing between 
* Null string, testing for 
e Null variable assignment, avoiding 
e Quoting 


within fest brackets 


to preserve whitespace 
* ryalue 


e Setting to null value 
e [n subshell not visible to parent shell 


e Testing a variable if it contains only digits 

* Typing, restricting the properties of a variable 
* Undeclared, error message 

* Uninitialized 

* Unquoted variable, splitting 

* Unsetting 

* Untyped 


wait, suspend script execution 


* To remedy script hang 


Weak quoting " ... 


while loop 
while [ condition ]; do 


* C-style syntax 
e Calling a function within test brackets 
* Multiple conditions 


* Omitting test brackets 
e while read construct 


Avoiding a subshell 


Whitespace, spaces, tabs, and newline characters 


e STFS defaults to 


* [nappropriate use of 
* Preceding closing limit string in a here document, error 


* Preceding script comments 
* Quoting, to preserve whitespace within strings or variables 


e [:space:], POSIX character class 
who, information about logged on users 


ew 
* whoami 
* logname 


Widgets 
Wild card characters 


* Asterisk * 
e[n [list] constructs 


e Question mark ? 
e Will not match dot files 


Word splitting 


e Definition 
* Resulting from command substitution 


Wrapper, shell 


x cK ck 
xargs, Filter for grouping arguments 


e Curly brackets 

e Limiting arguments passed 

* Options 

* Processes arguments one at a time 
* Whitespace, handling 


ok cK ok 


* Emulation 


ok ok k 
-Z String is null 
Zombie, a process that has terminated, but not yet been killed by its parent 


Prev Home 
ASCII Table 


